Craig, 44, said he used a day off from shooting new 007 movie "Skyfall" to film the sequence and as a result he was not on top form, reported Daily Star.
"She was fun, incredibly game. We had a short space of time and I was a bit grumpy as it was my day off and suddenly I am at the Palace with the Queen at her private chamber.
"And she improvises a little bit. She was supposed to be sitting at the desk and she asked if she could write, so she pretended to write," he said.
The actor was filmed in character visiting Buckingham Palace to meet with the royal before climbing into a helicopter together and appearing to parachute into London's Olympic Stadium during the sporting extravaganza in July.