Landowner Legal and General has refused to respond to the project by Oliver Laws, for the Raven Group, which is in competition with an office proposal by Eric Parry for the site.
The Raven Group first offered £65 million for Fortress House and was asked to come back with a larger bid. The developer returned with an offer of £85 million but claims to have heard nothing in response.
Oliver Laws was the firm behind the recent renovation of the state rooms at 10 Downing Street.
The practice's scheme aims to retain the non-listed building and turn it into a hotel. This is in direct competition with a proposal by Eric Parry to demolish it.
EH left the building last month after being paid £3 million to move out before the end of its tenancy contract. The move also came shortly after the quango successfully recommended the early-20th-century building be given immunity from listing.
The Oliver Laws proposals are being backed by SAVE Britain's Heritage, which has called for the building to be retained. But it is expected to be demolished sometime this summer.
'It is a fine building which especially well illustrates the use of Classical dimensions in the Early Modern movement and we thought it should be listed,' SAVE secretary Adam Wilkinson said.
'We are particularly keen on the contribution it makes to the surrounding conservation area.
'It is odd that Legal and General have not responded to the £85 million offer. They haven't even come back and asked for more money,' he added.