“Where’s Andy Coulson when you need him “ joked one Tory MP to me yesterday. At least I think he was joking. Yes, it’s got that bad. Downing Street has, at times, seemed rudderless over the last ten days, as it has been buffeted by various political squalls, which, added together have led political commentators to dub the post budget period as the worst political week of David Cameron’s life. Since the last one, I suppose. But Andy Coulson would know what to do. He’d know how to get a recalcitrant press back on side. Just as importantly, he would have also spotted the dangers of the Granny Tax and ‘Pastygate’.

On the face of it, the row over VAT on pasties is a ridiculous little spat over nothing very much. After all, fish and chips already attract VAT. Why didn’t people complain about the imposition of VAT on Rotisserie chickens? Answer: Because they’re seen as a bit posh.

We all know that reality isn’t as important as perception, especially in the grubby world of knockabout party politics. And the perception has grown that too many of the Tory frontbench are from another world – a world without pasties but a world full of foie gras. But there’s another row on the horizon, the imposition of VAT on static caravans. I am told it will raise £40 million but will result in losses to the exchequer of £45 million. Seriously.

It’s all very well clearing up tax anomalies, but if you’re going to annoy hundreds of thousands of people and take in less money, is it really worth the aggro? Would George Osborne even know what a static caravan was, and what it was for? The impression given this week by the Chancellor was of someone out of touch, flailing around and of someone who really thought this was all rather beneath him.

Add to this the shambles in communications over the tanker drivers’ strike and the backwash from the party funding scandal and you have a toxic cocktail of political battles.

I yield to few in my admiration of Francis Maude, a man who has been at the centre of the drive to modernise and detoxify the Tory Party, but I think even he would have to admit that this week hasn’t been his finest hour.

At the beginning of the week he argued that the PM need not reveal who he had entertained at Downing Street on the basis that his donor tete a tetes were “kitchen suppers”. In an instant he reinforced the image of incredible poshness. After all, who uses the word ‘supper’ nowadays, apart from the Downton classes? And the very phrase belies an assumption that you have a kitchen large enough to entertain in.

This impression of otherworldliness cannot be allowed to take hold and Tory backbenchers are well aware of it. To allege that they are in insurrection mode might be overstating the case, but even loyalist MPs are concerned. They worry that the prospect of an outright victory at the next election is slipping away. They had just about come to terms with the fact that most of them hadn’t got jobs in government. Now they are worrying that they won’t have a job in Parliament after 2015.

A rebellious parliamentary party is something any Tory leader must avoid at all costs. Ask Margaret Thatcher. Ask John Major. Ask IDS. The consequences can be calamitous.

Since 2005 George Osborne has built up a reputation as a formidable political strategist. Tory MPs are now beginning to question this accolade. They point to the fact that he was a key advisor to William Hague, he was one of the people who briefed Iain Duncan Smith for PMQs. They remind you that he was Michael Howard’s election strategist and then performed the same task for David Cameron. On neither occasion did his campaigns bring ultimate victory. It was he, it is said, who insisted that the European Referendum vote in the House of Commons was whipped. And it was he who failed to spot the political dangers in alienating Britain’s pensioners over the so-called ‘Granny Tax’. The fact that on this issue he has right on his side isn’t relevant. The last thing you want on the evening of your budget is the Director Saga on the airwaves denouncing your plans and accusing you of raiding pensions. If there were a share market in ‘Osbornes’, it would have lost half its value over the last ten days.

The one saving grace for Cameron is that Ed Miliband keeps missing open goals. This week he appeared at a branch of Greggs alongside Ed Balls, where they proceeded to buy 8 sausage rolls. Not pasties, we note. Miliband looked like he had never visited such an establishment before.

Many Tories believe that if Miliband has Alastair Campbell on his team, the Tories would be in real trouble. They think back to the latter years of the Major government when Campbell skilfully mixed a potion of sleaze out of Tory sex scandals and general fat-cattery. The charge stuck and Major ever recovered. Miliband’s team need to constantly ask: “What would Alastair do?” and then get on and do it.

All governments go through tough mid term patches and it was inevitable this one would too. It was also inevitable that at the same time the Liberal Democrats would run for cover. And true to form they have. Watching Sarah Teather on Question Time on Thursday twice protest that she couldn’t comment on much because “I am a government minister” left most of the audience incredulous. The LibDems are behaving as egoists fighting their own corner, but perhaps we shouldn’t be surprised. This has been historically true of a ‘factional’ party going right back to the days of the Whigs.

So, what should the Tories do? Firstly, they should acknowledge they have no one who seems to be able to deal newspapers in the way that Andy Coulson used to. His replacement, Craig Oliver, is adept at creating broadcasting opportunities, but many Tory MPs are frustrated that this ability doesn’t seem to transfer to the print media.

Secondly, the Prime Minister should bring forward his long awaited reshuffle. It is to Cameron’s credit that he doesn’t constantly chop and change his ministerial team, but it is self-evident that it needs freshening up. He needs to think very carefully who he promotes. People have had enough of the former special adviser clones. David Davis and Mark Pritchard were right this week when they called for the Tories to look more like the country which elects them – less of the posh gobs, more horny handed sons and daughters of toil. Expect to hear much more from the likes of Roads Minister, Mike Penning, Therese Coffey, Nick de Bois, Tracey Crouch, Alec Shelbrooke and Jessica Lee.

It is no coincidence that the very moment the PM’s closest strategic adviser Steve Hilton quits, is the very time things start to go wrong. Furthermore, David Cameron needs to clip the wings of Jeremy Heywood, who, it seems can do no wrong in his eyes. Heywood’s political antennae are not as acute as they could be but with the departure of Coulson and Hilton he reigns supreme in Downing Street. He now needs to be counterbalanced by a top level political appointment in the mould of Alastair Campbell – someone with a political brain who’s also a bit of a bruiser – and someone who won’t be afraid to say ‘No, Prime Minister’.

This government is at a turning point. It is crucial that the Prime Minister gets a grip. The next election can still be won, but the personnel decisions David Cameron makes in the next few months could well determine whether it will be.