How did the Social Democrats end up in such chaos?

The Social Democrats’ malaise goes far deeper than the expenses scandal engulfing leader Håkan Juholt, political scientist Nicholas Aylott writes, arguing that the party needs to decide its ideological path and to pick its leader accordingly.

• Wheelchair fraudster guilty of benefit heist (23 Mar 13)
• Swedish men trail women in care league (8 Mar 13)
• Reinfeldt 'more fun at dinner' than his foe (6 Jan 13)

The agony of the Swedish Social Democratic Party just gets worse. Until a year ago, the party had managed with just six leaders since the 1930s; it is soon likely to be looking for its third within a few months.

The latest incumbent, Håkan Juholt, put on a brave performance at Wednesday's party leader debate in Sweden’s parliament the Riksdag, but he surely cannot recover sufficient authority to retain his job for long.

The immediate cause of Juholt's predicament might induce déjà vu among British observers.

Late last week, the left-leaning tabloid Aftonbladet revealed that he had over-claimed accommodation expenses to which a member of parliament is entitled – an eerily similar offence to that which snared so many MPs at Westminster two years ago.

Juholt himself confesses to no more than severe carelessness.

But when exactly he became aware of the discrepancy is now – to his party's excruciating embarrassment – the subject of a preliminary investigation by the Prosecution Authority (Åklagarmyndigheten).

Despite the denials, the issue that all but seals Juholt's fate, however, is the fact that his expenses are only the latest in a remarkable series of blunders since he emerged from obscurity to become party leader only last spring.

His tendency to make hasty, ill-considered policy soon became all too apparent when the party’s line on Sweden's contribution to the Western intervention in Libya over the summer changed frequently and wildly.

More recently divisions within the party became all too apparent when Juholt's draft shadow budget caused uproar in the party's parliamentary group when it turned out to have little in common with the leader's earlier left-wing signals.

Further compounding the sense of disarray, party statements on the ultra-sensitive topic of immigration were spectacularly bungled and a television debate between party leaders was boycotted on flimsy grounds.

Media commentators speculate that leading Social Democrats may have leaked the expenses stories, calculating that short-term calamity might be worth enduring if it ends Juholt's disastrously misjudged leadership.

The question for a political scientist has to be: how did it come to this? How did what is still Sweden's biggest party, which nearly monopolised government for so long, end up in such chaos?

European social democracy faces all sorts of long-term challenges. But I think that the Swedish party's travails have a lot to do with its own institutions, and particularly how it chooses it leaders.

Swedish parties, like many Swedish organisations, are wary of internal competition for leadership positions. When a vacancy arises, the task of filling it is thus usually delegated to a selection committee (valberedning), which, having conferred and consulted, will frequently recommend only a single candidate for broader confirmation.

This can be a perfectly effective method of choosing leaders. But the Social Democrats have persisted with it, and in an extreme form, in entirely unsuitable circumstances.

This is a party that desperately needs an open, frank debate about its direction. Should it shift towards the centre and chase middle-class voters? Or should it rediscover what some would see as its ideological core and move left instead?

Either option is quite conceivable and legitimate. The party just needs to decide which to take.

Choosing a leader would, in most other Western parties, offer an excellent opportunity to do just this. Party members would simply opt for a candidate that championed one path or the other.

In Sweden, however, aspiring Social Democratic leaders are, by tradition, not supposed to pitch for the top job. Instead, they should wait for the party, in the form of the selection committee, to call.

This process means that since the Social Democrats lost the 2006 election, party members have never had the chance express their views in a leadership ballot – or perhaps even to decide what their views really are.

Equally bad, the leader who they eventually got in March was probably quite unprepared.

Rather than bridging the Social Democrats' ideological factions, the selection committee seemed paralysed by them, with each of the main leadership contenders was consistently blocked by the party's left or its right.

Juholt's emergence as a compromise figure, at almost the last minute, probably left him as amazed as anyone else. His party should not have been surprised, then, if he didn't have a thought-through vision of what he would do as leader, nor the managerial skill to implement such a vision. Until then, he had never really needed either.

The old selection method worked fine when the party was in government. What the party wanted when it changed leader was clear enough. A manifesto had previously been agreed by the party, endorsed by the electorate and formed the basis of a government programme. The new leader, always an experienced cabinet minister, was already signed up to it.

In opposition, however, the mandate for a new party leader is much less obvious. The previous election manifesto is, in practice, void after its rejection by voters. A different type of choice by the party is necessary. The Social Democrats' failure to change their way of choosing helps to explain why the last two selections, both made in opposition, have turned out so badly.

In fact, there is a simple way forward for the party.

Without abandoning the selection committee model entirely, two other Swedish parties have recently chosen new leaders in a more open way, with different candidates competing openly and offering distinct pledges on where they wished to take their parties.

Doing something similar would force the Social Democrats to decide what they want and, by definition, produce a leader in tune with that decision. Will this 122-year-old party be bold enough to break with its past in this way?