The 71-year-old former ODA chairman had originally said the reason behind his departure was because he wanted to concentrate on his construction company in the US.
But Lemley has now admitted in a newspaper interview with US regional newspaper the Idaho Statesmanthat he had become increasingly frustrated with political rows, slow progress and spiralling costs.
'I went there to build things, not sit and talk about it,' said Lemley. 'So I felt it best to leave the post and come home. A football field is not compatible with an athletics stadium.'
Lemley stated that he feared his reputation would be tarnished, as it was looking increasingly unlikely that the ODA would be able to prevent the project running behind schedule.
He said: 'In any event, there was a huge amount of politics. Those are the kind of things that confuse and frustrate the process.'
Lemley, who was the man behind delivering the Channel Tunnel, said he was not about to lose his record of delivering projects on time, and that this was the first time he had quit a construction project.
'I felt it was better to come home now than face that in five or six years,' he said.
The comments will come as a particularly damaging blow to the government following its claims last week that the Olympics project was running ahead of schedule, stating it wouldn't be another 'Wembley fiasco'.
In response to Lemley's comments, the ODA said in a statement: 'Jack Lemley set out the reasons for his resignation when he left the Olympic Delivery Authority.
'This project has been defined by strong political co-operation from the outset and that remains the case. It is a strength of the project that we have been having a debate about long-term legacy use of 2012 facilities now - rather than years in the future.
'That is what makes London 2012 unique. By any analysis we have made a strong start and hit all our major milestones.'
To see the original article visit: www.idahostatesman.com