The pair were travelling in a car in the heavily patrolled Hada district of south Sanaa, where several embassies are located, when they were intercepted by the kidnappers, the sources said.
There was no immediate word on the diplomat's nationality or the identity of the woman.
Witnesses reported that the diplomat's car, apparently armoured, was sandwiched between a van and a taxi behind it.
The diplomat and the woman stepped out of their car to argue with those aboard the two vehicles that had blocked their way. Gunmen then emerged from the taxi, seized the pair and drove them off in the two vehicles.
More From This Section
Hundreds of people have been abducted in Yemen in the past 15 years, the vast majority of them by disgruntled tribesmen who use the hostages as bargaining chips with the central government.
Nearly all of them have been freed unharmed after short periods of captivity.
But in recent years, Al-Qaeda's Yemen affiliate, regarded by Washington as its most dangerous, has brought a new, more threatening twist to the kidnappings.
The group has been holding a South African teacher since May last year. It is also still holding a Saudi deputy consul kidnapped in the southern city of Aden in 2012.