 "A Wall St. sign is seen outside the entrance of the New York Stock Exchange in New York's financial district")
Data showed that the private sector added the most jobs since June as markets awaited clues from Yellen on an expected interest rate increase on Dec 16.
Yellen will speak before the Economic Club of Washington at 12:25 p.m. ET (1625 GMT). She also testifies on the economic outlook before a joint committee of Congress on Thursday.
The economic data has not been playing ball with the Fed's policy plans in recent weeks and has been mixed at best. While construction spending has risen, there has been a sharp downturn in the manufacturing surveys along with weaker retail sales.
However, job growth has been strong and Friday's employment report is expected to show that the U.S. economy added 200,000 jobs in November.
Data on Wednesday showed U.S. private employers added 217,000 jobs in November, up from the 196,000 in October. Economists polled by Reuters had expected an addition of 190,000 jobs. The data could be an indication of the more comprehensive non-farm payrolls expected later in the week.
"Today's data bodes well for Friday's jobs report and the hawks in the Fed," said Adam Sarhan, chief executive of Sarhan Capital in New York.
Sarhan expects some sideways trading ahead of Yellen's speech, saying, "Investors are waiting for more cards to come out of the deck and with the ECB meeting tomorrow and the Friday jobs number, there are a lot more data points to digest."
The European Central Bank is expected to ramp up its trillion-euro bond-buying program on Thursday.
At 9:32 a.m. ET the Dow Jones industrial average was down 25.45 points, or 0.14%, at 17,862.9, the S&P 500 was down 2.34 points, or 0.11%, at 2,100.29 and the Nasdaq Composite was up 0.82 points, or 0.02%, at 5,157.12.
Nine of the 10 major S&P 500 sectors were lower with the energy index's 0.59% fall leading the decliners.
The CME Group's analysis of 30-day Fed funds futures prices shows investors place the probability of an interest rate hike this month at 75%.
A host of Fed speakers make appearances through the day, including San Francisco Fed President John Williams, Federal Reserve Board Governor Daniel Tarullo and Philadelphia Fed President Patrick Harker.
Atlanta Fed President Dennis Lockhart said there is a compelling" case for an initial hike in interest rates during the Fed's next meeting.
U.S. stocks started December on a positive note as health and consumer shares bounced back while auto sales suggested upbeat growth in November.
Yahoo shares were up 3.8% at $35 on Wednesday after reports that the company was weighing a sale of its core Internet business. Alibaba, in which Yahoo has a stake, was up 0.6% at $84.50.
Zafgen was down 61.1% at $6.18 after a second patient died during its obesity drug trial.
Aeropostale was up 3.5% at 56 cents ahead of reporting its results after the close of market.
Declining issues outnumbered advancing ones on the NYSE by 1,688 to 854. On the Nasdaq, 1,180 issues fell and 906 advanced.
The S&P 500 index showed 10 new 52-week highs and 3 new lows, while the Nasdaq recorded 24 new highs and 12 new lows.
