The Cabinet approved a bill to revise the 110-year-old law and wants parliament to pass it as quickly as possible, officials said.
Victims and their supporters have long claimed that the current law, which took effect in 1907, is out of date and that penalties are too lenient.
"As people's interest in the revision of penalties on sex crimes is high, we want to do our best to pass it swiftly," Justice Minister Katsutoshi Kaneda told reporters.
A victim will no longer have to file a complaint before prosecution for sexual crimes can occur. The current requirement is seen as a major reason for the underreporting of rape cases.
More From This Section
Rape victims for the first time will include men.
The number of rape cases in Japan recognised by police declined to some 1,200 in 2015 from more than 2,000 a decade ago, according to police.
But data from a Justice ministry study group found that nearly 75 per cent of sex crime victims do not seek criminal punishment for perpetrators.
Shinichi Kumada, an official with the Victim Support Center of Fukushima in northeastern Japan, said many victims feel that current punishment is insufficient.
"The revision does not mean all sex crimes will be eradicated, but I hope it will deter potential cases," Kumada added.