The Better Environmental Vision for Edge Lane (BEVEL) group - which has been opposing the proposed demolition of 360 terraced houses - has claimed that Prescott has predetermined the outcome of a planning inquiry.
If this accusation could be proven, it would put the veteran cabinet minister in an extremely precarious legal position.
BEVEL founder Elizabeth Pascoe, a local architect, has written to Prescott's department calling for a different minister to decide the outcome of the planning inquiry.
Pascoe argues that Prescott's passion for the Pathfinder programme means that he will have lost all impartiality when he decides whether to support or reject the planning inspector's recommendation. Just what he has recommended is currently top secret.
Prescott's main problem is his vociferous rejection of a recent SAVE Britain's Heritage report into the Pathfinder programme, a document which heavily featured the Edge Lane controversy, officially called the New Heartlands scheme.
In the letter, Pascoe makes the argument clear. 'His [Prescott's] rejection of recent criticism of the Pathfinder, and his statements made in a recent visit to Liverpool, show that he is not assimilating the inspector's report,' the letter says.
'In place of Prescott, we request another minister, able to retain a lawfully compliant stance in respect of the question of land acquisition against the will of the landowner, at the inquiry.
'[We want a minister who will] make the decision in the Edge Lane for the sake not only of our own homes, but our built heritage in its broadest terms,' it adds.