Gordon Bunshaft's personal house in East Hampton looks like it will be demolished after Stewart gutted the house in anticipation of Pawson's work.
But when Stewart's legal problems mounted in 2003, she put the house on the market. It failed to sell at first. However, earlier this year textile magnate Donald Maharam bought it for $9 million (£4.9 million).
It has since emerged that Maharam intends to demolish the house and rebuild on its footprint.
The businessman's architect, the Massachusetts-based David Pill, has been reported as saying that 'to restore the house would mean rebuilding it', and has also cited extensive problems with the building's foundations.
The move comes as a blow for fans of Bunshaft's work. The architect is well known for designing buildings such as Lever House (1951-52), the Manufacturer's Trust Company (1953-54), Connecticut General Headquarters (1954-57) and the Pepsi Cola Company Headquarters (1958-59).
Dennis Sharp, co-chair of conservation group Docomomo UK, said: 'We were horrified that this important building is likely to go. It's seriously threatened but so are so many others in that area.
'It's the same thing that is happening in the home counties - where people are buying up large sites to redevelop them, regardless of who designed any original building.'