Kapur's 71 on the third day comprised five birdies and six bogeys and he is now four-over 214 for three days.
Meanwhile, up on the leaderboard, Martin Kaymer (72) slipped a little after two successive 65s, but he was still five shots clear of the field.
At eight-under, Kaymer was five ahead of Rickie Fowler and Erik Compton, both of whom shot 67 each and were the only players in red numbers.
Kapur said, "I am striking the ball very well and the putting has also been fine. The course is set up really tough even without the roughs, but the pin positions and the greens are a challenge.
Also Read
"That's how it is in the Majors and it is a new challenge every round and I am really liking it here at this level."
Speaking of the round, he added, "It's been solid and after a series of birdies and bogeys that last birdie felt very good and it puts me in a right frame of mind for Sunday."
Jeev's tied ninth at the 2008 PGA Championships is also the best by an Indian at any Major.
Kapur had a bogey-bogey start after the slopes took the ball away from the greens on a day when the greens were tough and pin positions very tricky.
But he birdied three in a row from fourth to sixth and added a fourth birdie on the ninth.
His driving and iron play was superb and he gave himself a lot of chances getting inside 10-12 feet and holing those birdie putts.
But the course and luck went against him as he dropped four bogeys in five holes from 12th to 16th.
In at least two cases the ball again slid off the slope and pushed him to bogeys.
The birdie on the 18th ensured he would leave the course with a smile and in a positive frame of mind to come back and make a good charge on the final day.