Андрей Солдатов - The Red Web - The Struggle Between Russia's Digital Dictators and the New Online Revolutionaries

While the scientists pursued his theory, Fant began to be concerned that law enforcement would abuse speech recognition technology. Fant’s concerns were confirmed in 1970, when FBI director J. Edgar Hoover went to Stockholm and gave an interview to the local newspaper Dagens Nyheter about shining prospects for using voice samples for identifying terrorists. When the newspaper asked Fant for comments on Hoover’s remarks, Fant cautioned that the method he had developed was imprecise and it was premature to use to identify anyone. His rebuttal was so surprising that the newspaper printed on the front page photographs of Hoover and Fant opposite each other, presenting him, as Fant put it later, as a “possible FBI enemy number one.” [7] Fant’s autobiography on the site of the Department of Speech, Music and Hearing of KTH Royal Institute of Technology in Stockholm, founded by Fant. Gunnar Fant, “Half a Century in Phonetics and Speech Research,” www.speech.kth.se/gunnarfant/halfcentury.pdf .

Soviet scientists had no such reservations. Research centers working on speech recognition opened in many cities, and the section on acoustics at the Academy of Sciences coordinated the nationwide research. But everybody knew that the true boss was the KGB.

An instrumental part of the research was in Leningrad, the Scientific Research Institute of Dalny Svyazi, or of long-distance communications, known as Dalsvyaz. This is the facility where Sergei Koval, a graduate of the physics department of the University of Leningrad, began work in 1973 on acoustics. He was always interested in the science of sound, but what was also attractive was a promised monthly salary bonus of 15 percent. He was unconcerned that the institute was shrouded in secrecy. The offices of his applied acoustics unit were always guarded by men with automatic weapons. The institute, with more than ten thousand personnel, was overseen by a ministry for industrial telecommunications, but its real purpose was to work for the military. The applied acoustics unit of three hundred people that Koval joined was not under control of the institute at all but was instead run by the KGB, who paid these additional personnel the bonus. It was a classic Russian matryoshka —secrets within secrets—applied to research.

Koval soon realized the reason for such secrecy. His colleagues told him that this unit was in fact the Marfino sharashka that had been transferred to Leningrad. One day he was pointed to a bespectacled engineer who worked at a neighboring laboratory. His name was Valentin Martynov, and he had once served in Marfino along with Kopelev and Solzhenitsyn (and featured in Solzhenitsyn’s novel The First Circle under the name “Walentulya” Pryanchikov). Koval recalled that Martynov was “meticulous and stubborn.” A young, enthusiastic engineer of the late 1940s, Martynov had remained devoted to speech recognition for decades. He went so far as to defend a thesis for a degree on the topic. Although he was free now, he still walked every day into the territory guarded by the men with automatic weapons and dogs to do research for the same secret services that had once sent him to prison. Koval never tried to ask him why: “It was a generation that was much more mature. It was not suitable to talk about the past.”

By the 1970s Koval’s applied acoustic unit became the main coordinator of research funded by the KGB in speech recognition. He recalled, “There was a section of applied problem-solving at the Academy of Sciences. This section took orders and research commissions on prospective research from all the agencies, from the Ministry of Defense and from the KGB. The section demanded money, and it always got the money. The scheme was wonderful: the money then was allocated to the applied research departments belonging to the KGB, like our department. So we were able to distribute this money right across the different academic institutions as we saw fit. We could effectively sponsor any project we wanted. I myself was the curator of the scientific program, where forty universities were involved.” [8] Sergei Koval, interview with authors, January 2012.

What began in the 1940s with seven people in the acoustic laboratory in Marfino had, by the 1970s, become a sprawling, well-funded empire of secret research. There was no clear line between KGB-sponsored research and civilian research; it was all part of the same empire. The secrecy touched everyone—numbering millions of people. It showed up in the most unexpected places. For example, the Computation Center of the Academy of Sciences on Vavilova Street in Moscow, which Ed Fredkin loved to visit in the 1980s to talk personal computers, was one of the research institutes quietly working for Koval’s unit.

Vladimir Chuchupal joined the section of voice recognition of the Computation Center in 1980. He was told that the main task of the section was to apply computers to speech recognition. Chuchupal was warned that it was strictly prohibited to mention to anyone outside the Center the name of their main “customer”—the Dalsvyaz and the KGB. He was put in direct contact with Kuchino almost immediately. Chuchupal knew exactly what they worked on—one day his chief described how he was given notes from the legendary Kopelev to study. [9] Vladimir Chuchupal, interview with authors, September 2014.

Thanks to the generous funding provided by the KGB, in the early 1980s Chuchupal’s section got its first personal computers, some Soviet-made machines and a few IBM PS2s. When they arrived, the issue of speech recognition opened up a vista for surveillance the KGB had never imagined possible—applying computer technologies to phone tapping meant that not only could a speaker be identified but that what he said could be used to trigger the interception system (the surprising byproduct of the project was the computer game Tetris, designed on one of the KGB computers). That, at least, was the theory. The KGB came up with the idea of using key words so that mention of “the bomb” or the “Communist Party,” or anything else chosen by the KGB and put in the system would automatically initiate interception of the phone line. This option could have changed the KGB’s modus operandi completely—in most previous cases the KGB needed to know the identity of the suspect to start eavesdropping on his phone; now the technology would provide the suspects. But it was also very challenging; the keyword system was an ambition, but making it a reality depended on more computing power than was available.

The Soviet research empire into speech recognition along with Dalsvyaz and the unit at the Computation Center worked actively on the issue for years. Once again, there was no clear line between the civilian and KGB research.

This empire cracked with the collapse of the Soviet Union, but it didn’t go away. Initially the KGB cut its research programs. “In 1990 our funding stopped,” Koval recalled, who himself left Dalsvyaz. “Two-thirds of employees quit immediately.” But he did not leave the field of research. With his laboratory chief, Mikhail Khitrov, and five colleagues, he founded a private company that in 1993 became the Speech Technology Center, trademarked in the United States as SpeechPro. Each of eight founders got an equal share of 12.5 percent of the company. The speech recognition scientists tried to succeed with civilian contracts; for example, they developed a talking book for the Society for the Blind.

But soon their old friends, the security services, returned. Koval’s company got its first contract from the Interior Ministry to build a system of using phonoscopy for chasing criminals. Then the FSB offered a contract to make a system that would separate voice from background noise. In the 2000s the company employed up to 350 people—roughly the size of the original Soviet department in Dalsvyaz. “I cannot say what kind of work we do for them,” Koval said of the security services, “but it all continues, it’s the same—what we did then, we do now.”