The actor played the part of Mammy's son-in-law Dino Doyle in the comedy series and its predecessor stage show for more than two decades but alleges that he was abruptly dropped from the most recent festive specials after querying his pay.
Hollywood is now in the process of suing for unfair dismissal and in a new interview has compared O'Carroll, who has members of his real-life family playing most of the main cast, to the "mafia".
"So much of the show, off screen, is about Brendan's own family. It's like jobs for the Browns," Hollywood told the Daily Mail.
"At times, as the years went on, it felt like it was a case of the Mrs Brown's mafia, which was difficult if you weren't in it. There is a fear of falling out with them, and of course that creates a tense atmosphere and I found myself walking on eggshells."
He continued: "I am so hurt by what Brendan has done to me. The fame and fortune has gone to his head. I worked with him for 20 years... and this was not how I thought my time working on Mrs Brown's Boys would end. I'm devastated.
"What I have learned is it is his way or the highway. In my opinion, Brendan has changed so much in recent years since becoming more rich, famous and powerful.
"When I raised my problems with Brendan, I was met with hostility. I felt it was bullying... like you can't say what you want to any more. I was told I should be grateful, that I'm lucky."