Nine of the 10 major S&P sectors were lower, with the interest-rate sensitive utilities sector's 3 percent decline easily the worst. The financials sector, up 1 percent, was the only gainer.
The Labor Department's report showed nonfarm payrolls increased by 271,000 in October, beating the 180,000 expected. Data for August and September were revised to show 12,000 more jobs on average were created than previously reported.
The unemployment rate fell to 5.0 percent, the lowest since April 2008, from 5.1 percent in September. The jobless rate is now at a level many Fed officials view as consistent with full employment.
"I think it's good news - it's good news for the economy, eventually the market will take it as good news," said Sean Lynch, co-head of global equity strategy at Wells Fargo Investment Institute in Omaha, Nebraska.
"It's a blowout number, it's a strong number for the jobs and the consumer should be feeling pretty good heading into the last couple of the months of the year."
The dollar rose to a 6-1/2 month high after the data.
More From This Section
Higher rates increase borrowing costs for companies, while a rise in the dollar hurts their income from overseas markets.
At 9:56 a.m. ET (1456 GMT), the Dow Jones industrial average was down 52.18 points, or 0.29 percent, at 17,811.25.
The S&P 500 was down 10.18 points, or 0.48 percent, at 2,089.75 and the Nasdaq Composite index was down 16.18 points, or 0.32 percent, at 5,111.56.
Chicago Fed President Charles Evans, an FOMC voting member, said the payrolls numbers are "very good" and that he will keep an open mind at the next Fed meeting on Dec. 15-16.
Traders raised the odds of a hike in December to 70 percent from the 58 percent just before the jobs data was released, according to the CME Group's FedWatch program.
JPMorgan led the rise in financial stocks, rising rose more than 3 percent, giving the biggest boost to the S&P 500.
Goldman also rose 3 percent and was the biggest influence on the Dow, followed by Disney, which was up 2.5 percent after strong results.
TripAdvisor fell 10 percent to $75.02 after quarterly results missed estimates.
Weight Watchers rose 23 percent to $20.82 after it raised its 2015 adjusted profit forecast.
ZS Pharma soared 41 percent to $89.07 after AstraZeneca said it would buy the biotech company for $2.7 billion.
Declining issues outnumbered advancing ones on the NYSE by 2,241 to 628. On the Nasdaq, 1,509 issues fell and 916 advanced.
The S&P 500 index showed 10 new 52-week highs and five new lows, while the Nasdaq recorded 96 new highs and 38 new lows.
(Reporting by Abhiram Nandakumar in Bengaluru, additional reporting by Charles Mikolajczak; Editing by Savio D'Souza)