Luke Harding - The Snowden Files

The NSA has had its own operations branch at GCHQ Cheltenham since the 1950s, as well as in London; GCHQ staff work at MHS. With some advance warning other GCHQ employees from Cheltenham can visit the heavily protected US outpost.

The NSA has a senior US liaison officer attached to the UK intelligence community known as SUSLO; his British counterpart operating in Washington under diplomatic cover is called SUKLO. Lesser GCHQ employees are assigned to practically all NSA facilities; they are called ‘integrees’. There is even a GCHQ staffer at the NSA’s tropical base in Hawaii, where Snowden worked.

Typically GCHQ employees do at least one stint at an NSA facility. The agency provides a helpful glossary for the Brits on American life; it gives tips on car hire and points out that in the US a boot is a ‘trunk’. There are joint meetings, training courses, exchange visits, cryptological workshops and celebratory dinners. And, one suspects – though Snowden’s documents don’t tell us this – the odd inter-agency romance.

This intelligence-swapping arrangement dating back to 1947 has been a success story. One document speaks of ‘another fine example of NSA and GCHQ working well together’. The Anglo-American SIGINT partnership is often warm on a personal level, beneficial to both parties and historically enduring. You might call it a marriage.

The files, meanwhile, offer a rare insight into the cloistered world of British spying. Salaries of GCHQ staff may be low but the organisation offers its linguists and mathematicians lots of leisure activities: pub quiz nights, cake sales, trips to Disneyland Paris and an internal puzzle letter called Kryptos . It even has its own social networking site, SpySpace. The main drawback to a GCHQ career is the agency’s provincial location. ‘Be prepared to describe where Gloucestershire is,’ a GCHQ recruitment guide says.

One particularly sensitive aspect of TEMPORA is the secret role played by telecoms companies which own or manage the fibre-optic cables. GCHQ calls them ‘intercept partners’, liaison with whom is handled by ‘sensitive relationship teams’. They include some of the world’s leading firms. BT, the main intercept partner, is codenamed ‘REMEDY’, Verizon Business ‘DACRON’, and Vodafone Cable ‘GERONTIC’. Four smaller providers also have codenames. In 2009, Global Crossing was ‘PINNAGE’, Level 3 ‘LITTLE’, Viatel ‘VITREOUS’ and Interoute ‘STREETCAR’.

Between them these companies help intercept most of the cable links touching the UK. They have British landing points at Lowestoft, Pevensey Bay, Holyhead (linking the UK to the Republic of Ireland), Whitesands Bay, Goonhilly and other seaside towns.

The company names are classified even higher than top secret, as ‘Strap 2 ECI’ – ‘exceptionally controlled information’. Exposure might presumably lead to customer unhappiness. One leaked document warns of potential ‘high-level political fallout’ if the firms’ identities become public. Intelligence sources stress that the companies have no choice. As in the US, they can use the excuse that they are compelled by law.

Thanks to this corporate co-operation, for which the telecoms companies are paid substantially by the British taxpayer, GCHQ was handling 600 million ‘telephone events’ a day by 2012. It had tapped more than 200 fibre-optic cables which touched the UK. It was able to process data from at least 46 of them at a time. This is indeed a lot of data – more than 21 petabytes a day – and the equivalent of sending all the information in the British Library 192 times every 24 hours.

Yet inside GCHQ there is still anxiety that the organisation will fall behind. One of the team responsible for managing TEMPORA sets out how the agency’s ‘mission role’ grew. New techniques had given GCHQ access to huge amount of new data or ‘light’ – emails, phone calls and Skype conversations. ‘Over the last five years, GCHQ’s access to “light” [has] increased by 7,000 per cent.’ The amount of material being analysed and processed had increased by 3,000 per cent, he said – an astonishing figure. The agency was ‘breaking new ground’ but also struggling to keep up. ‘The complexity of our mission has evolved to the point where existing management capability is no longer fit for purpose.’

An internal review for 2011/2012 also warns: ‘The two major technology risks that GCHQ has to face next year are the spread of ubiquitous encryption on the internet and the explosion in the use of smartphones as mobile internet devices. Over time, both of these technologies could have significant effect on our current tradecraft.’

The agency predicts that by 2015, 90 per cent of all internet traffic will come from mobile phones. There were already 100 million smartphones around the world in 2012. The mobile was the ‘most prolific customer product ever invented’. GCHQ was launching a new project to ‘exploit mobile devices’, the document said. It meant ‘getting intelligence from all the extra functionality that iPhones and BlackBerrys offer’. GCHQ’s end goal was: ‘to exploit any phone, anywhere, anytime’.

TEMPORA and allied projects may be impressive. But in inventing them, the western espionage agencies seemed oblivious to the larger picture: that the state was now indiscriminately collecting the communications of millions of people, without their knowledge or consent.

In the past, British spooks attached crocodile clips on copper wires to eavesdrop on the phone calls of thieves and villains or Irish Republican terrorists. These were individual targets approved on individual ministerial warrants: the identifiable bad guys. Now, though, the NSA and GCHQ were hoovering up data from everyone on a Brobdingnagian scale. This included data from a majority of people who were entirely innocent.

Officials insist they don’t have the analysts to sift through all this private correspondence. One told the Guardian : ‘The vast majority of the data is discarded without being looked at… we simply don’t have the resources.’ He said: ‘If you had the impression we are reading millions of emails, we are not. There is no intention in this whole program to use it for looking at UK domestic traffic – British people talking to each other.’ The head of GCHQ, Sir Iain Lobban, publicly repeats the spies’ favourite analogy of a ‘vast haystack of data’, containing needles.

The haystack does, of course, consist of the communications of both Britons and foreigners. GCHQ’s mass sweepings included among other things the contents of cables linking the international data centres belonging to Google and Yahoo, where they passed across British territory.

The British spies quote obscure UK legislation dating from 2000, which permits unrestrained foreign intelligence-gathering. They say this Regulation of Investigatory Powers Act (RIPA) allows them to bulk-collect all ‘external’ internet communications. ‘We turn somersaults to obey its spirit and letter,’ one said. The word ‘external’ is interpreted – some would say twisted – to mean anything tapped from a cable that has at least one foreign end. Because of the way internet links work, this means that anyone in Britain who sends an email is often also talking to GCHQ. Not something the ordinary paying customer who signs up to BT and Google can find on their contract, even in the very smallest print.

Both the British and the Americans can make secret searches inside this ‘haystack’ of mass data for patterns of behaviour, for contact chaining of groups of friends and for target individuals. Secret letters signed by British foreign secretaries – the first was Labour’s David Miliband in 2009, the next the Conservatives’ William Hague – apparently authorise queries made with a view to investigating foreign political intentions, nuclear proliferation, terrorism, serious financial crime and the UK’s ‘economic wellbeing’. How is this policed? Government lawyers have since demonstrated in British cases that the word ‘terrorism’ is capable of being interpreted very widely.