Harding, Luke - WikiLeaks - Inside Julian Assange's War on Secrecy

Other reporters included Washington correspondent Ewen MacAskill and Latin America correspondent Rory Carroll in Caracas. (Carroll’s VPN connection quickly packed up, making it impossible to eyeball the Chávez cables.) Simon Tisdall, Ian Black and Jonathan Steele, all immensely experienced, combed through the cables on the Middle East and Afghanistan. The sheer range of journalistic expertise that five major international papers were throwing at the data would perhaps demonstrate the value of the world’s remaining MSM. They could be the genuine information professionals, standing out in an otherwise worthless universe of internet froth.

Sitting in the fourth-floor bunker, Harding and a colleague, reporter Robert Booth, were among those who would spend long hours staring, increasingly dizzy eyed, at the dispatches. It soon became clear that there was an art to interrogating the database. If your search term was too big – say, “Britain”, or “corruption” – the result would be unfathomably large. The search engine would announce: “More than 1,000 items returned.” The trick was to use a relatively unusual name. Better still was to experiment with something off the wall, or even a bit crazy. Putting in “Batman”, for example, yielded just two results. But one was a delightful cable in which a US diplomat noted that “Dmitry Medvedev continues to play Robin to Putin’s Batman.” The comparison between the Russian president and his prime minister would whizz round the world, and prompt a stung Vladimir Putin to accuse the United States of “arrogance” and unethical behaviour.

Likewise, punching in the search term “vodka” popped the cork on unexpected results: drunken meetings between US ambassadors and central Asian despots; a memorable wedding in Dagestan in which Chechnya’s president – the murderous Ramzan Kadyrov – danced with a gold-plated revolver stuck down his trousers; and a Saudi Arabian sex party that spoke volumes about the hypocrisy of the Arab state’s princely elite.

In contrast to the staccato jargon of the war logs, the cables were written in the kind of prose one might expect from Harvard or Yale. Harold Frayman had improvised the original search engine used to sift the much smaller Afghan and Iraq war logs. By now he had improved these techniques. “I’m a journalist. I knew what we were going to look for,” he explains. “Diplomats were much more verbose than squaddies in the field. They knew longer words.”

The data set contained more than 200 million of those words. Frayman had originally used the computer language Perl to design the Afghan and Iraq databases. He describes it as a “very well developed set of bits of software … It did little jobs very tidily.” For the cables Frayman added refinements. Journalists were able to search the cables sent out by individual embassies. In the case of Iran, which had not had a US mission since the 1970s, most of the relevant diplomatic chatter actually came out of the US embassy in Ankara. It was therefore helpful to be able to quickly collect up the Ankara embassy output.

Of the files, 40% were classified confidential and 6% secret. Frayman created a search by five detailed categories: secret/noforn (that is, not to be read by non-Americans); secret; confidential/ noforn; confidential; and unclassified. There was no top-secret: such super-sensitive material had been omitted from the original SIPRNet database, along with a substantial number of dispatches that the state department in Washington considered unsuitable for sharing with its colleagues in the military and elsewhere. There were idiosyncrasies in the data: for example, very little material from Israel seemed to be circulated: suggesting that the US embassy there did not play an intimate role in the two-way dealings between Tel Aviv and Washington, and was largely kept out of the loop.

“Secret” was the place for the rummaging journalists to start. Some of these searches produced remarkable scoops. Many, however, did not. The secret category, it soon emerged, tended to cover a limited number of themes: the spread of nuclear material and nuclear facilities; military exports to Iran, Syria and other states considered unsavoury; negotiations involving top-ranking US army personnel. By far the largest number of stories came from lower classified documents.

Like the other reporters, Harding and Booth soon found themselves developing their own quirky search techniques. They discovered it was often useful to start at the bottom, working backwards from a country’s most recent cables, written as they were up to 28 February 2010. Such searches became, however, an exercise in stamina; after reading a batch of more than 40 cables, the reporters had to take a break. Adjacent to the secret bunker was a free coffee machine. There was also a relaxation room. “Here, after a long session of cable-bashing, you could at least flick the sign to engaged, grab a cushion and lie groaning on the floor,” says Harding. Katz said the company would pay for massages: but none of the Guardian ’s weary cable slaves had time to spare.

To editor-in-chief Alan Rusbridger, the abundant disclosures pouring out of the US cables at first seemed like a player hitting the jackpot every time in an amusement arcade. He recalled how Leigh – after reading the material for a couple of weeks over the summer, chortling and astonished – had come back with enough stories for 10 splashes, articles that could lead the newspaper front page. “It was a fruit machine. You just had to hold your hat under there for long enough,“ Rusbridger says.

The analogy is a good one. But it perhaps makes the task sound too easy. To comb properly through the data, teams of Guardian staff had to be recruited. The reporters, especially the foreign correspondents, brought much to the table: contextualisation, specialist knowledge and a degree of entrepreneurship in divining what to look for. All these skills were needed to turn the cables into significant newspaper stories.

Leigh sent a memo to Rusbridger:

We’ve now got to the stage of story selection on project 3. The previous exercises (Iraq and Afghanistan) worked well politically, I thought, because Nick and I were able to focus the coverage [and the resultant global coverage] on elements that it was highly in the public interest to make known.

With Afghanistan, this was civilian casualties. With Iraq, it was torture. This time, I think it’s also important that we try and major on stories that ought to be made known in the public interest. That was the compass-needle which helped me when I originally tried to put together the first dozen stories.

So – top stories revealing corruption and crime (Russia, Berlusconi, etc) and improper behaviour (eg unwarranted US pressure on other countries, unwarranted leaking to the US by other country officials). This will then position us where we can be best defended on all fronts??

A herd of publishable articles began to grow in size. The task of readying them for publication fell to Stuart Millar, the Guardian ’s web news editor, who says he felt like a harried cowboy. “I was trying to lasso them into some kind of shape.” This was a far more complicated production problem than the similar exercise with the Iraq and Afghan war logs. At first, it had seemed the cables would yield just a hatful of stories. By the eve of D-day, Guardian journalists had produced more than 160 articles, with more coming in all the time. “There was a crazy, enormous heave of copy,” Millar recalls.

For Millar, as a web expert, it was clear that the emergence of the vast cables database marked the end of secrecy in the old-fashioned, cold-war-era sense. “The internet has rendered that all history,” he reflects. “For us, there was a special responsibility to handle the material carefully, and to bring context to the stories, rather than just dump them out.”