"Late yesterday, we have seen limited Russian troop activity in the vicinity of the border with Ukraine that may suggest that some of these forces are preparing to withdraw," Rasmussen said in Montenegro.
"It is too early to say what this means, but I hope this is the start of a full and genuine withdrawal," he said.
"At present, most of the previously deployed Russian force remains near the Ukrainian border and we see continued Russian exercises in the same area."
"This would be a first step from Russia into the right direction of living up to its international commitments, especially as Ukraine is preparing to hold important presidential elections on Sunday."
Also Read
President Vladimir Putin on Monday announced that Russian troops near the border, estimated by NATO to number 40,000, were to return to bases after the end of spring exercises.
Their presence had raised deep concerns after Moscow's annexation of Crimea in March and an uprising by pro-Moscow rebels in eastern Ukraine.
Russia's defence ministry said Thursday that four trains and more than a dozen planes had taken equipment and troops away from the area.