All 33 subindexes were lower on Monday, the first trading day of the new fiscal year, with exporters, financials and real estate stocks among the big losers.
The Nikkei dropped 2.1 percent to 12,135.02, the biggest one-day decline in a week. The index added 19.3 percent in January-March, marking its best quarterly performance since April-June 2009.
Now, it stands 4.1 percent below a 4 1/2-year high of 12,650.26 hit on March 21.
"Short-term foreign investors were selling today, as they decided to take profits as they saw that hopes for bold monetary easing are mostly priced into the market," said Makoto Kikuchi, the chief executive of Myojo Asset Management.
"That is not to say that sentiment is negative. People are still hopeful about easing, and we see some buying by long-term investors like pension funds that are raising their stance to 'neutral' from 'underweight' on Japan."
Toyota Motor Corp dropped 2.1 percent, Sony Corp tumbled 4.3 percent, and brokerage Nomura Holdings <8604.T> shed 4.3 percent.
More From This Section
Railway stocks, which have high asset value, weakened, with East Japan Railway Co shedding 5.8 percent and West Japan Raiway Co dropping 7.2 percent, which was also hit by a brokerage downgrade.
Warehouse shares also lost ground, with Mitsui-Soko Co nosediving 8.1 percent and Shibusawa Warehouse Co shedding 5.2 percent.
Tankan
The losses came even as a Bank of Japan tankan survey showed that business sentiment in Japan improved for the first three months of 2013 due to expectations for the central bank to ease monetary policy.
Big manufacturers' mood improved after two straight quarters of deterioration, with the headline sentiment index rising 4 points to minus 8, in line with a median market forecast of minus 7.
The survey comes ahead of the BOJ's first policy-setting meeting under new Governor Haruhiko Kuroda on April 3-4, when the board is set to expand monetary stimulus and debate an overhaul of its policy framework.
"The tankan result was not remarkable, but if the result were too good, it could spur worries that the central bank may not have to ease policy drastically, so the result was after all good for the market," said Hikaru Sato, a senior technical analyst at Daiwa Securities.
The broader Topix declined 3.3 percent to 1,000.57, falling below its 25-day moving average of 1,017.94. Volume was thin, with 2.85 billion shares changing hands, compared with last month's daily average trading volume of 3.24 billion shares.
The Nikkei has gained 16.7 percent this year, outperforming its global peers. By comparison, the Dow Jones Industrial Average has added 11.3 percent, the Standard & Poor's 500 Index has gained 10 percent while the pan-European FTSEurofirst 300 index has advanced 4.8 percent over the same period.
Helped by the rally, Japanese equities now carry a 12-month forward price-to-earnings ratio of 14.2, a level not seen since July 2010 though still below the 10-year average of 16.4, according to Thomson Reuters Datastream.
"Investors are expected to buy exporters again when their outlooks for this fiscal year will be out next month," said Myojo's Kikuchi, adding that the Nikkei will likely continue outperforming its global peers.
Bucking the weakness on Monday, Tokyo Electric Power Co surged as much as 12.6 percent before ending up 4.3 percent after the utility company announced an additional cost-cutting plan for the two fiscal years ending March 2015.