I've been there too and found I was penalised by larger firms when I took a career break, no matter how short! However I have since found the perfect solution.... I work in the domestic sector with normal clients...(not footballers wives) and love what I do. By systemising everything, I find I can have the right work life balance. And spend the majority of my time designing infront of clients!
I have since set up Pride Road, a supportive network to help architects in a similar situation.