Netherlands coach Guus Hiddink blamed Kazakhstan's early goal for the problems his side faced breaking down the visitors before three goals in the last half-hour secured their first win of the Euro 2016 qualifying campaign.
The Netherlands dominated possession but surprisingly fell behind when Renat Abdulin's diving header from a corner in the 17th minute gave goalkeeper Jasper Cillessen no chance.
The Kazakhstan defence was like a brick wall that held firm until goals by substitute Klaas-Jan Huntelaar, Ibrahim Afellay and a Robin van Persie penalty earned victory in Group A.
"We were thrown back by their only corner of the game. After the opening goal they lined up in a (wall-like) defence," Hiddink told reporters.
"Then it is up to you to break through by executing a high-speed passing game, but that didn't always work. It is important not to panic. If you keep playing like that the goals will come sooner or later."
The final 30 minutes proved Hiddink right after a tactical change that was the turning point in the match.
The hosts started with a traditional 4-3-3 after the 5-3-2 system former Netherlands coach Louis van Gaal successfully used at the World Cup in Brazil did not work in their opening Euro 2016 qualifier against the Czech Republic which they lost 2-1.
Extra striker
Hiddink added another striker when Huntelaar came on for defensive midfielder Nigel de Jong in the 55th minute.
The Schalke 04 captain paired up with national team skipper Robin van Persie up front and rewarded the coach by heading in a free kick six minutes after coming off the bench.
Moments later Kazakhstan were reduced to 10 men when striker Baurzhan Dzholchiyev was dismissed for a tackle from behind on Afellay, who put the Dutch ahead with only eight minutes left to cap his return to the national team after 23 months.
The Olympiakos Piraeus midfielder had been plagued by injury in the previous two seasons Which kept him out of the team.
"Afellay has had a difficult time. It is good to see he has found the way back up. I think he played with a great drive today," said Hiddink.
Van Persie wrapped up the win with a penalty in the 89th.
Hiddink has only four days to determine whether he will stick to a 4-3-3 formation with their next European Championship qualifier against Iceland scheduled for Monday.