Geller told the AJ he had lambasted 'his friend' Pei shortly after the building opened in 1989 to criticise him for stealing 'his idea'.
The Israeli-born spoon bender said: 'I lived next to IM Pei and we still talk about pyramid power. After he designed the Louvre I rang him up and said he stole my idea. I have now built one in my garden.'
Meanwhile, the enterprising Geller is now in talks with Lower Mill Estate developer Jeremy Paxton over loaning his art collection to a new gallery to be built at Paxton's high-profile development in the Cotswolds.
He continued: 'I know Jeremy, the owner of the Lower Mill Estate - he is very interested in art. I have worked with Salvador Dali, Andy Warhol and paint myself.'
Lower Mill Estate has already featured work by some of the country's most famous architects, including Piers Gough and Will Allsop.
Pei was unavailable for comment at the time of going to press.