The proposal is for a mixed-use development which includes retail and residential property, a hotel, new transport links and large leisure facilities - but major parts of the project now face wholesale amendments.
In its comments the CABE review panel said, 'We think the project team should reconsider the design of the hotel, which does not relate well to the Garrick Theatre.
'We do not think it appropriate for the proposed hotel to dominate the theatre, and feel that the form currently envisaged is alien in this location. We think that the height of the hotel should be reconsidered.'
Another area of the project which will need to be reconsidered is the design of the residential facilities opposite the hospital.
The report stated: 'The way this steps in plan at the corner between Birmingham Road and St John Street does not seem in keeping with the character of Lichfield.
'A more straightforward approach, with buildings addressing the street as elsewhere, could be more successful.'
The comments will come as a blow to Chetwood's £40 million project, and the practice will probably be sent back to the drawing board by Lichfield district council, which is hoping to hold on to as much of the city's history as possible.