"I informed parliament's legal affairs committee that I have started a preliminary investigation over tapping of a mobile phone of the chancellor," Harald Range said.
The long-anticipated probe, which follows an explosive allegations last year that US spies had eavesdropped on Merkel's mobile in the past, is against unnamed persons, Range said after addressing the committee.
However he said he had decided against opening an investigation into claims of wider NSA surveillance against German citizens.
Data-sensitive Germans reacted with outrage to the accusations by fugitive intelligence contractor Edward Snowden amid sensitivity over mass state spying on citizens by the Stasi secret police in the former communist East.
Also Read
A parliamentary panel has been established to assess the extent of spying by the NSA and its partners on German citizens and politicians, and whether German intelligence aided its activities.
US President Barack Obama sought to quell the international furore over the reports, announcing in January that he had halted spy taps on friendly world leaders and curtailed the reach of mass NSA phone surveillance.
In an interview with a German TV channel a day later he assured Merkel that he would not let intrusive surveillance harm their relationship, but said intelligence gathering on foreign governments would continue.