Luke Harding - The Snowden Files

The debate turned into one of the most impassioned for years; speakers for and against the amendment were applauded from the aisles. Leading against Amash was Mike Rogers, a former FBI agent, the chair of the House intelligence committee and a straight-talking NSA defender. ‘Have we forgotten what happened on September 11th?’ he asked. He mocked an online campaign backing Amash, and said: ‘Are we so small we can only look at how many Facebook likes we have?’ Republican Tom Cotton, speaking against the Amash proposal, declared: ‘Folks, we are at war.’

But some members opposed to warrantless surveillance invoked comparisons with colonial days. They likened the NSA’s programs to the general warrants that allowed British customs officers to search private property. This was about the most emotive charge that could be laid by an American politician. (The lawyer for Snowden’s father, Bruce Fein, made the same resonant comparison in a TV interview to those British ‘writs of assistance’.)

The debate found strange bedfellows. Ted Poe, a leading member of the Tea Party, united with liberal Zoe Lofgren, something that nearly never happens in Washington. But Nancy Pelosi, the top Democrat, whipped against the Amash amendment. Feelings were high. During the debate Rogers scowled and smacked his rolled-up papers into his empty hand like a truncheon, pacing the rows of desks. Amash was laughing – this was a career-making moment for him – and joking with colleagues.

When it came, the vote was a shock. The amendment was defeated, but only just – by a margin of 217 to 205. Few had anticipated that dissatisfaction in Congress had reached this level. It reflected a polarisation across America. The country was engaged in full-on debate. For some, it was security versus privacy. For others, it was whether Snowden was a whistleblower or a traitor. There were those who thought it mattered and those who didn’t.

For the White House, the NSA and the Office of the Director of National Intelligence the vote was a near-death experience. It was clear that something had to change. The absolutist mantra that Snowden was a ‘little traitor from Hawaii’, as Alexander put it, was no longer enough. The White House began to hint at compromise. Congressional hearings were pencilled in for the autumn; there were calls for legislative change to curb the NSA; work began to frame new bills.

In his press conference before the summer break, on 9 August, Obama made his first substantial remarks on the crisis. He laid out a strategy of greater transparency. But, crucially, he didn’t announce any restrictions on the surveillance.

Obama proposed a new panel to review intelligence policies. He also announced greater oversight of FISC, the foreign intelligence surveillance court, and the declassification of the legal rationale that underpinned the collection of phone records under section 215 of the Patriot Act.

The president acknowledged the US had ‘significant’ spying capabilities. But he said that unlike other, repressive regimes it behaved with restraint, and didn’t throw its ‘own citizens in prison for what they say online’. His reforms, he said, were designed to ensure Americans could trust US intelligence efforts and have confidence they were ‘in line with our interests and our values’.

He had a message, too, for non-Americans, a subspecies under US surveillance laws with no apparent privacy rights whatsoever. ‘To others around the world, I want to make clear once again that America is not interested in spying on ordinary people.’

All of this sounded reasonable. But sceptics wondered whether Obama meant reform or ‘reform’. In other words, a simulacrum of reform in which the NSA’s more egregious bulk surveillance practices would be allowed to carry on unhindered. In late August the new review panel was unveiled. Obama had promised a ‘high-level group of outside experts’. These ‘independent’ experts, it turned out, were virtually all former intelligence officials with close links to the Obama administration.

Civil libertarians sniffed a large rodent. The panel’s chair was Michael Morell, Obama’s former deputy CIA director; two other members included Richard Clarke, a former counter-terrorism co-ordinator under Clinton and George W Bush, and Peter Swire, Clinton’s privacy director. The panel enjoyed the lugubrious name, ‘Director of National Intelligence Review Group on Intelligence and Communications Technologies’. In it was a clue: the advisers were working out of the offices of the DNI, headed by James Clapper. The committee’s report – to be written by the end of 2013 – went to the White House.

Critics dismissed the panel as pretend transparency, and its members as White House stooges. This may have been unfair. But it was hard to tell, since the panel’s meetings were conducted in secrecy. In September it held an inaugural session with civil liberties groups including the ACLU. Another hearing followed with representatives from Facebook and other tech giants, still reeling from the PRISM disclosures.

Silicon Valley lashed out at the White House. Executives from Facebook, Google, Microsoft, Apple and Yahoo all said the Snowden revelations had been a disaster for their businesses, with their Europe- and Asia-based operations distinctly harmed. Billions of dollars had been lost. The administration needed to get a handle on the situation and do something expeditiously, the tech giants said. This conversation took place before it emerged that the NSA had hacked into Google and Yahoo’s data centres – in effect, a state cyber-raid on two major US firms.

Throughout the summer the tech companies had pumped out the same message: that the NSA was coercing them, legally, to co-operate. Any data they handed over wasn’t done voluntarily, but in response to a court-approved stick-up. A few days before their appearance at the review panel, Silicon Valley CEOs had gathered at the TechCrunch Disrupt Conference in San Francisco. The mood was mutinous. Yahoo’s Marissa Mayer said her company had to obey FISA court orders, even though it didn’t like them: ‘When you lose and don’t comply, it’s treason.’ Facebook’s Mark Zuckerberg put it succinctly. The ‘government blew it,’ he said.

During meetings with the review panel, however, the tech companies didn’t say anything about restricting NSA surveillance. Instead, some attendees suggest, the companies’ chief aim was to tell the customers a good story about how they were all protecting their data.

The news that the NSA had hacked Google and Yahoo’s data centres, however, proved a game changer. In their most concerted deed yet, the tech giants united to demand sweeping changes to US surveillance laws. In an open letter to Obama and Congress, they called for a ban on bulk data collection by spy agencies.

They wrote: ‘The balance in many countries has tipped too far in favour of the state and away from the rights of the individual – rights that are enshrined in our constitution. This undermines the freedoms we all cherish. It’s time for a change.’

The signatories were Apple, Google, Facebook, Microsoft, Yahoo, LinkedIn, Twitter and AOL. They were, naturally, acting in their own economic interests. But the firms also set out a series of five ‘reform principles’. Chief among them was that governments – the US, UK and the rest – should end suspicion-less surveillance. Instead of spying on everybody they should focus on ‘specific known users for lawful purposes’.

The Snowden revelations, Google added, were in danger of turning the internet into the ‘splinternet’. ‘The ability of data to flow or be accessed across borders is essential to a robust, 21st-century global economy,’ they argued.

In this new post-Snowden world, the NSA faced a full-blown public relations calamity. Since it was founded – appropriately enough in total secrecy – the agency had experienced four distinct epochs. The first was creation. It lasted from 1952 until 1978. The era ended with a series of reports by the Senate committee, led by Frank Church, into unforgivable domestic abuses: the FBI’s harassment of Martin Luther King, CIA assassination programs and the watch-listing of 75,000 Americans. The Church committee ushered in wide-ranging reforms. Among them was the Foreign Intelligence Surveillance Act, which established there must be court approval for foreign intelligence surveillance inside the US.