BERLIN, Sept 24 -- German Chancellor Angela Merkel won a fourth term in office on Sunday but Europe's most powerful leader will have to govern with a far less stable coalition in a fractured parliament after her conservatives haemorrhaged support to a surging far right.

Two years after Merkel left German borders open to more than 1 million migrants, the anti-immigration Alternative for Germany (AfD) stunned the establishment by becoming the first far-right party to enter parliament in more than half a century.

The AfD won 13.0 percent of the vote - more than expected and one of many shocks on a night of drama that saw Merkel's conservatives get their worst result since 1949, and her main Social Democrat (SPD) rivals their worst since 1933.

Describing the far right's success as a test for Germans, Merkel insisted she had a mandate to govern - a formidable challenge as she has little choice but to cobble together a three-way coalition with a pro-business group and the Greens.

"Of course we had hoped for a slightly better result," a humbled Merkel said after her conservative bloc slumped to 32.9 percent of the vote - down from 41.5 percent at the last election in 2013.

But she added: "We are the strongest party, we have the mandate to build the next government - and there cannot be a coalition government built against us."

The euro slipped around 0.4 percent in early Asian trading as it became clear the results would make forming a coalition tricky for Merkel.

Coalition building could take months as Merkel's only straightforward path to a majority in parliament would be a three-way tie-up with the liberal Free Democrats (FDP) and the Greens - an arrangement untested at national level.

The Social Democrats, who have served with Merkel's conservatives as junior partners in a "grand coalition" for the past four years, won just 20.6 percent of the vote, as nearly half of voters repudiated the two parties that have dominated Germany since World War Two.

SPD leader Martin Schulz said the party would refuse to rejoin a coalition and instead take up its position as the main opposition. The Social Democrats appear to have been hurt badly by being in government, making it difficult to distinguish themselves from Merkel's conservatives.

After shock election results last year, from Britain's vote to leave the EU to the election of U.S. President Donald Trump, leaders of Europe's establishment have looked to Merkel to rally the liberal Western order.

A pastor's daughter who grew up in East Germany, she has acted as an anchor of stability in Europe and beyond. Now, she faces an unstable situation at home as she must form a coalition, an arduous process that could take months.

'Tectonic shift'

Sunday's election, fought against the tense backdrop of surging support for far left and far right parties across Europe, delivered a fractured German parliament with six party groups, up from four previously.

Josef Joffe, publisher-editor of Germany weekly Die Zeit, said the vote marked a "tectonic shift in German politics" and that the three-way coalition Merkel looks likely to try to forge will be "highly unstable".

Leading AfD candidate Alexander Gauland vowed his party would "hunt" the new government, whatever its make-up, adding: "We'll get our country and our people back."

In France, far-right leader Marine Le Pen congratulated the AfD, tweeting: "Bravo to our AfD allies for this historic showing!"

A large group of protesters gathered outside the AfD's post-election party and police said they made several arrests.

The European Jewish Congress expressed alarm at the AfD's success, adding: "We trust that centrist parties in the Bundestag will ensure that the AfD has no representation in the coming governing coalition."

The AfD says immigration threatens German culture, but denies that it is racist: "We will neither tolerate xenophobia nor racist positions. But we simply don't have them either," AfD co-leader Joerg Meuthen said.

The result makes kingmakers of both the FDP and the Greens, both of which have played the role in the recent past but neither of which now has enough support on its own to give Merkel a majority.

FDP leader Christian Lindner, an ambitious 38-year-old who preaches an ultra-hard line on Europe and has unsettled the German political establishment, said he was open to coalition talks with Merkel but that Germany needed a change of course.

The Greens' Katrin Goering-Eckardt said: "We will see if there can be cooperation."

A three-way tie-up of Merkel's conservatives, the FDP and the Greens - known as a "Jamaica" coalition because the black, yellow and green colours of the three parties match the Jamaican flag - is widely seen as inherently unstable.

The Greens - keen on regulation - and the business-friendly FDP are at opposite ends of the political spectrum and a clash of policy visions would be likely on tax, energy, the European Union and migrants.

Despite losing support, Merkel, Europe's longest serving leader, will join the late Helmut Kohl, her mentor who reunified Germany, and Konrad Adenauer, who led Germany's rebirth after World War Two, as the only post-war chancellors to win four national elections.