The Cockburn Association and the Edinburgh World Heritage Trust (EWHT) believe the architect's plans will destroy the city's historic integrity.
But Murray has dismissed the mounting opposition to his scheme for the Scottish capital's Old Town and Canongate areas as 'anti-development pie in the sky'.
He said: 'These people simply fear change. Their alternatives would never get off the first base and exposes an ideological gulf.
'Their plan harps back to Medieval times and ignores the fact that Edinburgh is a 19th century city. But no-one is questioning whether this proposal makes sense.
'Poundbury or Poundstretcher seem to be the only options - this is untenable.'
The Murray masterplan, which is currently out to public consultation, includes a luxury hotel and conference centre, a new public square, offices, cafés, bars, restaurants and 300 homes.
But heritage bodies are alarmed at plans to bulldoze two listed buildings and two blocks of 1930s tenements to make way for a new road linking the Royal Mile with Waverley station.
EWHT director Zoe Clarke claims the city will lose a precious part of the World Heritage Site if Murray's scheme goes ahead.
She said: 'If it is approved in its current form, the World Heritage Site will be replaced by 'anywhere architecture'.
'The Old Town and Canongate have a strong sense of place and special character which has been built up over many generations, but regretfully the masterplan ignores it. I don't think that is what the people of Edinburgh want,' she added.
Even so the developer behind the scheme, Mountgrange, is claiming overwhelming public support for the project.
They say that a Mori opinion poll of 500 people in Edinburgh commissioned shows almost 80 per cent of those quizzed back the plans.