Rev Philip Lambert, the rector of Dorchester, said locals were prejudiced against residents of the high-profile village in Dorset.
The letter will come as bad news for the Prince's Foundation, which has attempted to claim that the scheme is a model for urban design techniques that should be used in the Thames Gateway. The village was built as an extension to Dorchester on land owned by the Duchy of Cornwall.
Writing in the Dorchester Parish Magazine, Lambert said some people had been against the development being built on farmland, while others believe 'posh' people live there.
'Some locals feel they are people who have come into the community, who have retired and who are perhaps fairly wealthy, and who are of a particular social set, or just posh is the word you would use,' he said in the letter.
Lambert continued that the reality was that houses, offices and shops were being occupied by 'all kinds of people from all backgrounds'.
`The buildings are designed imaginatively and the people are the same as the people elsewhere in Dorchester. But there is a problem,' he added. 'The problem is prejudice.'