Cameron made the statement at Prime Minister's Questions after Labour MP Joan Walley urged him to back a ban on pesticides harmful to bees.
Peter Chaunt, of Oxfordshire Beekeepers' Association, confirmed that Cameron was ex-president of the body but had resigned after he became the Prime Minister.
Cameron was quizzed by Walley, who is chair of the parliament's Environmental Audit Committee, in the House of Commons on Wednesday.
His reply, in which Cameron said he had been "neglecting his duties" at the association, led to some Labour MPs to shout "resign".
More From This Section
Chaunt said while Cameron's comment may have been a slip of the tongue, he had actually served as president of the association, which is based within his Witney constituency.
"When he became prime minister he wrote to us to say that with his new post he would be rather busy, and obviously beekeeping wasn't going to be at the forefront of his activities," Chaunt was quoted as saying by the BBC.
"He was coming up to the end of his period of presidency so we wrote back and thanked him very much because he'd been very helpful.
In a statement, Number 10 said: "The prime minister has a long and happy connection with the Oxfordshire Bee Keepers' Association.
"For some years he was honorary president. This position recently came to an end.
"The prime minister is a strong advocate of beekeeping in his constituency and as he said in the house it's important we look after our bee population."
Chaunt said he had written to Cameron informing him of the error, the BBC said.
The new president of the associaiton is John Craven.
The protest came ahead of a vote on a European Union proposal to ban the use of the chemicals.