The two-day meeting in Beijing came after diplomats held informal talks on the sidelines of a humanitarian conference in the Chinese city of Shenyang between Red Cross officials from the two countries earlier this month.
"We would like to have serious and frank discussions over a broad range of outstanding issues for both sides," Junichi Ihara, head of the Japanese Foreign Ministry's Asian and Oceanian Affairs Bureau, told Japan's NHK public broadcaster.
Song Il-Ho, North Korea's ambassador for talks to normalise relations with Japan, replied: "I completely feel the same way," Japan's Kyodo news agency reported.
At the North Korean embassy, the venue of the first day of the meeting, Song expressed hope that relations between the two countries will start moving "in a positive direction," Kyodo said.
More From This Section
Song also compared the resumption of governmental dialogue to the arrival of spring, "when icy rivers melt and water begins to flow," Kyodo reported.
The meeting comes amid recent mixed signals from Pyongyang over its willingness to re-engage in diplomacy with Tokyo.
The talks were officially called off in December 2012 when Pyongyang launched a long-range missile, drawing international condemnation. Formal ties with Japan could bring huge economic benefits to the impoverished state.
North Korea outraged Japan when it admitted more than a decade ago that it had kidnapped 13 Japanese in the 1970s and 1980s to train its spies in Japanese language and customs.
Five of the abductees have been repatriated to Japan along with their families. But Pyongyang has insisted, without producing solid evidence, that the eight others are dead.
Yokota, who was taken while on her way home from school, has remained a painful symbol of the abduction issue. Her parents' meeting with their granddaughter was welcomed as a step in the right direction.
Japan's Prime Minister Shinzo Abe hailed the "change" in Pyongyang's attitude, which eased off its earlier demands that any meeting should be held in North Korea.