Mueller's kids are currently being looked after by Sheen's other ex-wife Denise Richards, reported People magazine.
Richards has two daughters, Lola and Sam, with Sheen, and adopted another daughter, Eloise, on her own. She has cared for Mueller's children on several occasions when Mueller was battling with drug in a rehab.
Sheen supports the arrangement and has often gushed about Richards being "a stable, very maternal" influence on the twins.
"He loves those kids. But he's got a crazy work schedule, filming two shows a week and would rather they spend time with Denise and their daughters than with a nanny at his empty house."
"Brooke is angry that Denise may take the kids. She's decided she wants the boys to be with her brother and sister-in-law," the source added.