The 59-year-old said he feels there is "nothing" he can do to improve his label and should call it quits on his career.
"More now, actually. I suppose there were moments then, but really, now. I have this wonderful company, and I have these amazing people whom I taught the business, and they're better than me. The pupils are better than the teacher," Madden told Marie Claire magazine.
"And as the teacher and the builder, I have to figure out another way that I can add value to my company. So now, because I'm an entrepreneur, maybe there's nothing I can do anymore. Maybe I should just quit... I have thought that. But then I have incredible days where I feel valuable.
Disclaimer: No Business Standard Journalist was involved in creation of this content