The retired four-star Marine general, accompanied by a US delegation, was welcomed by Salman at Al-Yamama Palace in Riyadh, telling the monarch: "It's good to be back."
Mattis, who commanded troops during the 2003 invasion of Iraq, arrived in the kingdom on Tuesday afternoon to listen to Saudi leaders and learn "what are their priorities", an American defence official said earlier.
The United States and Saudi Arabia have a decades-old relationship based on the exchange of American security for Saudi oil.
Saudi leaders felt Obama was reluctant to get involved in the civil war in Syria and was tilting toward Riyadh's regional rival Iran.
More From This Section
The Sunni Muslim kingdom "felt marginalised" during international negotiations on a nuclear accord with Shiite Iran, the defence official said.
That deal, signed in July 2015 by the Obama administration, saw the lifting of international sanctions in exchange for guarantees that Tehran will not pursue a nuclear weapons capability.
Mattis was also to meet Crown Prince Mohammed bin Nayef and Salman's son, Deputy Crown Prince Mohammed bin Salman, who holds the defence portfolio.
Saudi leaders worry about Iran interfering in Arab countries by using Shiite communities to advance their pawns, as in Bahrain, Lebanon and Yemen.
Bordering Saudi Arabia, Yemen has been torn for more than two years by a civil war between Iran-backed Huthi rebels, their allies, and pro-government forces aided by a Saudi-led military coalition that receives some military support from the US.
The Saudis have found a more favourable ear in Washington under Trump, who denounces Iran's "harmful influence" in the Middle East.
Mattis has called Iran "the single biggest state sponsor of terrorism in the world".
The US military is watching Huthi activities along the strategic Bab al-Mandab strait connecting the Red Sea with the Indian Ocean.
Yemeni rebels in late January attacked a Saudi warship in the Red Sea, and they are also believed to have fired missiles towards US warships in the area.