Philip Webster - Inside Story - Politics, Intrigue and Treachery from Thatcher to Brexit

That was a reference to Osborne’s 2012 Budget when what seemed like a decent package on the day collapsed quickly with retreat after retreat on matters such as taxes on Cornish pasties and caravans. It became known as the ‘Omnishambles Budget’.

But when, on the early evening of 18 March 2016, Downing Street received a letter from IDS resigning over the disability cuts, a shocked Cameron realized this was deadly dangerous to him. He called IDS, asking him to hold off until they had spoken face to face. Soon government sources were briefing that the £1.3 billion cuts were being ‘reviewed’.

But IDS had had enough, and in a second telephone call told Cameron his mind was made up. During furious exchanges, Cameron was reported to have called IDS a ‘shit’, something that neither side has seen fit to deny. Duncan Smith’s friends maintain he had been surprised by Osborne’s decision to put the cuts in the Budget so that they could be counted as billions of savings in the deficit battle, but that the last straw was to put them in the same package as tax cuts for the better off. ‘It went against Iain’s whole social justice message, and he had to go,’ an insider said.

IDS had been in constant rows with Osborne over the previous six months about the chancellor’s plan to cut tax credits for three million people, eventually defeated by the Lords and dropped, and the IDS plan to merge several benefits into a universal credit. One friend said: ‘Iain regrets not going earlier. He thought about resigning before Christmas 2015 and wishes he had. He tried to defend the Budget but lost heart and realized he could not in all honesty do so.’

IDS, for decades an opponent of the European Union, has insisted since that his resignation had nothing to do with Europe, and he stayed out of the campaign for a few weeks to show that. But his enemies maintained that the whole exercise was designed to damage Cameron and Osborne at a time when they could least afford it. Some claimed that IDS had planned to quit the Government dramatically during the Budget debate, something his friends have denied.

I can confirm, however, that he was one of several ministers who called on Cameron privately earlier in the year to allow ministers freedom to speak out during the referendum campaign. IDS told him that if he did not grant the concession, ministers would resign and that would be far more damaging to the Government. I understand that the key figure in persuading the PM to give way – much to the unease of key pro-Europeans like Michael Heseltine – was Chris Grayling, leader of the House of Commons.

By the end of 2015, Grayling had concluded that the deal the Prime Minister had been negotiating with Europe would not be good enough to change his view that Britain would be better off out. He decided that he would campaign to leave but delayed until the New Year before telling Cameron. After the regular 8.30 a.m. meeting of ministers, aides and Commons business managers on Monday, 5 January, Grayling stayed on for a private chat with the PM. He told him that he intended to campaign for an ‘Out’ vote and offered to resign. On the same day, Theresa Villiers, the Northern Ireland secretary, had a similar conversation.

Cameron had been moving towards allowing ministerial freedom as Harold Wilson had for Labour ministers in the 1975 referendum but he had not intended to announce it at this stage. While Grayling’s remarks were an offer to resign, Cameron would have seen them as a threat, and concluded that ministerial resignations would be more damaging than allowing them latitude. He may have concluded that keeping them inside the tent would avoid the more abrasive campaigning that would be inevitable if they were speaking from outside the Cabinet. He got that wrong. He was to be shocked by the interventions of Outers such as Gove, Johnson, Leadsom and others.

So why in the end did Cameron lose a campaign that he believed from 2013 that he would win and win well?

As they gathered on the morning before referendum day there was cautious confidence – much more than there had been for some time – among the leaders of ‘Britain Stronger In Europe’, the official all-party campaign to remain in the EU.

Andrew Cooper (Lord Cooper of Windrush), the founder of the polling company Populus and director of strategy for David Cameron between 2011 and 2013, had for a few days been bringing better news to the gathering of Downing Street aides (including Craig Oliver, the communications chief) and Labour and Lib Dem strategists. Less than twenty-four hours before the polls opened the PM was told that he would win by several points.

But the late confidence was misplaced because Downing Street and other campaigners had underestimated the impact of immigration on the campaign and overestimated the impact of the economy. The Tories had won the 2015 election on the back of economic competence and thought they could do it again. There appears to have been a basic mistake in the so-called ‘playbook’ on which the campaign was based.

The previous summer, on the basis of a survey involving thousands of respondents, Populus presented the board of ‘Stronger In’ with a finding that suggested that the economy was massively more important than immigration to most voters. The conclusion was not challenged and treated as a fait accompli , according to campaign sources. It meant that from the moment Cameron fired the starting gun, warnings about the impact of a Brexit on the economy flowed from the mouths of Chancellor, Prime Minister, Bank of England Governor and any half-respectable think-tank or international body, with the President of the United States pitching in to suggest that Britain would drop to the back of the queue in the negotiation of post-Brexit trade deals. Little had been prepared on immigration.

The sheer ferocity of the warnings from George Osborne – he threatened an emergency Budget in the final days of the campaign – appears in the end to have been counter-productive, with ordinary voters accusing ministers of going over the top and not believing what they were told in any case.

The survey finding may also have resulted in Cameron asking for and winning less from Brussels in his winter negotiations to tackle fears over immigration than his party or the country expected. Earlier suggestions that he would get changes to the EU’s freedom of movement rules did not materialize, and Cameron came back only with an agreement that migrants would have to stay in the UK for four years before being entitled to benefits. Whether Cameron would have insisted on more if he had been told that immigration was a much bigger priority for voters than he expected, we will never know.

Some within Number 10 felt Osborne was given too much prominence in the campaign and that his apocalyptic warnings came so often that they were not believed. In Cameron at 10: The Verdict , Anthony Seldon and Peter Snowdon said there was frustration among Cameron’s aides about the Chancellor’s conduct. ‘He had to be restrained several times,’ they say. They warned him that his reputation was often a hindrance. ‘This wasn’t a general election and it required diplomacy and subtlety.’

Others claimed Osborne and Cameron had to do more than they intended because Corbyn did not do enough big events on behalf of the joint campaign. Less than three weeks before the vote there was deep concern in the Cannon Street, London, headquarters of ‘Stronger In’ when Cooper reported that ‘Remain’ was behind for the first time and that immigration was drowning out the message on the economy.

Despite independent authorities – from the Bank of England to the International Monetary Fund – warning of the dangers of Brexit, the country seemed increasingly immune, apparently believing these organizations were doing the work of the ruling elite against whom they were about to rebel. Nothing infuriated Cameron more than Gove’s television claim on 3 June that ‘people in this country have had enough of experts’. ‘It was utterly irresponsible. Dave and Samantha had already had it with Michael by then. But this was the last straw,’ a friend said.