Twentieth Century Society director Catherine Croft described the decision to demolish the building as 'a pity', and compared it to the Baltic Arts Centre in Gateshead, Tyne and Wear – a former flour mill.
She said: 'It is a landmark building and could have been to put to good use. The Baltic Arts centre shows what can be achieved with a good conversion.'
The removal of the 62m-tall cathedral-shaped behemoth – owned by Associated British Ports – will take around 16 weeks.
Plymouth City Council said there were no applications for the vacant site.