The Twentieth Century Society (C20) called the destruction of the building's altar last week 'one of the greatest sins of conservation' in the UK.
Gillespie Kidd and Coia's category A-listed building has been neglected by its owner and the local authority, C20 claimed.
'Historic Scotland and the local authority, Argyll and Bute, charged with the protection of listed buildings, have so far not enforced its maintenance and repair,' C20 said.
The spokesperson for the conservation group continued: 'If St Peter's is not protected from vandalism, it might very soon be too late to think about reuse.
'Those charged with the preservation must act now to prevent the loss of a true masterpiece.'
And John Deffenbaugh, the secretary of the St Peter's Building Preservation Trust, agreed: 'The destruction of the high altar is symptomatic of the college's wider decline, which has accelerated in the last 12 months.
'Without immediate action, the college will soon be destroyed beyond recognition,' he warned.