The building was cordoned off late last week after a survey revealed that cladding on the Modernist tower block - built in 1961 - was structurally unsound and in danger of coming away from the building.
Plymouth City Council chief executive Barry Keel said: 'This inspection has revealed that the problems we were already aware of have worsened much more rapidly than expected.
'Some of the cladding panels are now beginning to come away from the building. We are now urgently carrying out further detailed inspections of the tower to establish what needs to be done and how best to make it safe.'
Keel denied that the timing of the inspection was coincidental and claimed that the survey was booked before the furore over the listing broke.
'However,' said Keel, 'it does confirm what we have been saying to English Heritage - the building has fundamental faults due to the way it was constructed and should not be preserved.
'We will do what we have to in order to make it safe but we do not want to spend money on maintaining a poor building instead of spending it on services for the people of Plymouth.'