The city's ever-powerful conservation lobby is gearing up to get the structure, built in the 1850s, spot listed as a way of getting the Wilkinson Eyre-designed proposals put on hold.
The factory, called the Newark Works, was designed by Thomas Fuller, an architect who is relatively unknown in the UK, but who in later life became the chief architect of Canada.
Fuller became extremely well known in North America after designing the New York State Capitol in Albany and a series of government buildings in Ottawa.
The demolition plans are sure to trigger the wrath of conservationists, with SAVE Britain's Heritage leading the calls to stop the bulldozers.
SAVE's Adam Wilkinson said: 'This is Fuller's best work in the UK and with its massive windows and careful detailing it is easily the best factory in Bath.
'It is shameful that it is threatened with demolition.
'The main facade could easily be incorporated in to any new plans for the site. Its demolition is entirely unjustified - if it was in North America it would almost certainly be protected.'
And a bizarre response from a government spokesman appeared to miss the point - with the spokesman apparently under the belief that the Fuller factory is an existing school building.
'Victorian school buildings have served generations of pupils and teachers very well but many cannot meet the needs of the end users in terms of physical accessibility and accessibility to a modern curriculum,' the spokesman said.
'Where cost-effective existing schools are modernised we have a unique opportunity to tackle a generation of under-investment in our schools and provide children and teachers with the 21st-century facilities they deserve.'