Deciphering the federal election outcome

But first, why believe my analysis? What right do I have to pontificate on this result?

Last week I wrote "if there's a widespread mood for change it's not showing up", drawing a direct analogy to the '93 result when "the opposition lost an unlosable election". I'd began writing that column assuming I could chart Labor's path to government - after all, I'd been continually told Labor would surge to victory. The problem was nobody could show how this would occur. The path wasn't clear.

Apparatchiks explained exactly why the opposition would win this seat or that seat, but always on a case by case basis. That's not the way swings work. The great insight of psephologist Malcolm Mackerras was individual electorate analysis can't be trusted. There's too much noise involved and it's never until everybody's filled in their ballot papers that the result's apparent. Focusing on the overall swing removes this uncertainty; it allows for both the grumpy individual voter and exceptional campaigning by specific MP's. It smooths out the discrepancies to allow an overview.

But there was no trend. So I threw away the jottings demonstrating Labor would win and noted instead, "disillusion is spiraling through the electorate spinning voters off into the arms of minor parties". It was obvious that Bill Shorten needed to harness the preferences of this "huge, disaffected minority". He couldn't.

All the opinion polls showed Labor ahead, although, critically, never by more than the margin of error. The obvious assumption was that the pollsters had their fingers on the scales. This was particularly the case when, in the final weeks, 17 polls from different organisations proclaimed the opposition either one or two points ahead. In a brilliant post, Mark the Ballot demonstrated why this was impossible. The point was that pollsters were somehow touching up their figures. Either they weren't confident or were deliberately skewing the result. It turned out that they simply couldn't work out where preferences would go. They were playing it safe.

On the day itself, in the ballot booths, Labor's primary vote collapsed. Secondly, the party failed to recognise that if it wasn't the first choice of voters it would need their preferences; yet there was no strategy to achieve this.

When it came to that final moment, standing with pencil poised, disillusioned voters just didn't believe in the alternative. It wasn't so much a failure of the pollsters, as a failure of Labor to inspire.

Nearly a quarter of the vote (24.7 percent) went to minor parties (up from 23.5 percent in 2016). Labor has only itself to blame for the fact that these people failed to give the party their second preferences, but that's what happens when you write people off. There was also an amazing 688,000 informal ballots, more than one in 20, many deliberately so. Why didn't the party inspire these voters?

These issues are critical, because they go to the true cause of the devastating defeat.

Labor's attitude was summed up in Chris Bowen's dismissive comment, telling self-funded retirees to "vote against us" if they didn't like Labor's policy on dividend imputation. Well, they did.

People weren't inspired enough to vote for change.

This isn't rocket science. The party took its program to the people on a take-it-or-leave-it basis and guess what? Voters left it. They don't like being lectured at from on high. People wanted to be participants in a conversation and feel as if they were being included in policy formulation as equals. Instead they were told what to do.

That failure to explain goes to Shorten's problem. He ignored the traditional National Press Club speech in the last days of the election. He thought he was appealing over the heads of the media and had already swung all the voters he needed. He hadn't.

We are still a society of the many - the aspirational as well as those without. Yes, we need change; urgently. However the way that will be achieved isn't by slamming citizens into a wall and lecturing them. Everyone's got to be brought along together.

The key can be found in Tasmania. There, two electorates of ordinary Australians, not wealthy, not part of 'the big end of town', voted to dispatch Labor parliamentarians. They plumped for - God help us - Scott Morrison instead. Shorten's rhetoric failed to connect with their needs and feelings. This is the key point.

While reasons for this defeat are myriad, these are not the cause. Many might not have trusted a leader who had played a key role in killing off, successively, both his own previous leaders (Kevin Rudd and Julia Gillard). Perhaps others were hit by the tax changes. Maybe some were nurses, or perhaps teachers, surprised that the party could announce significant childcare reform yet didn't appear to have any program addressing their concerns. But the true cause was the party presented a program without being prepared to discuss it.

It appeared arrogant and out-of-touch.

That's why Labor needs new leadership. People who will listen.