Mattis, a retired general seen as a moderating force on the often impulsive president, made little attempt to hide his disagreements with Trump.
“Because you have the right to have a Secretary of defence whose views are better aligned with yours," Mattis said in a letter to Trump, “I believe it is right for me to step down from my position."
Mattis hailed the coalition to defeat the Islamic State and also defended NATO, the military alliance between North America and Europe whose cost-effectiveness has been questioned by the businessman turned president.
“My views on treating allies with respect and also being clear-eyed about both malign actors and strategic competitors are strongly held and informed by over four decades of immersion in these issues," Mattis wrote.
The resignation came as The Wall Street Journal reported that Trump was also considering a substantial drawdown in a much larger US operation -- Afghanistan.
Trump, who has surrounded himself with former military men and was said to have been impressed by the scholarly and calm 68-year-old Mattis, was diplomatic as he announced the departure, writing on Twitter that the defence secretary achieved “tremendous progress."
Mattis will serve until the end of February, giving Trump a short window to name a successor who can be confirmed by the Senate.
Just hours earlier, Trump took to Twitter to bash a frequent ally known to covet the job, Senator Lindsey Graham, after the lawmaker also questioned the wisdom of exiting Syria.
Doubling down on his declaration on Twitter of victory over the Islamic State group a day earlier, Trump boasted of commanding the world’s most powerful military and said that others should lead the fighting.
“Does the USA want to be the Policeman of the Middle East, getting NOTHING but spending precious lives and trillions of dollars protecting others who, in almost all cases, do not appreciate what we are doing? Do we want to be there forever? Time for others to finally fight," he tweeted.
But he found little support. The top Republican and Democrat on the House Armed Services Committee, Mac Thornberry and Adam Smith respectively, came together Thursday to voice concern, saying a withdrawal would be a “strategic error" that would endanger Kurdish allies and empower rivals Russia and Iran.
“ISIS was and still remains a lethal threat to America’s security and will seek to inspire threats to the homeland from safe havens around the world," they said in a joint statement.
The United States will keep up air strikes in Syria so long as troops remain, Pentagon spokeswoman Commander Rebecca Rebarich said, but it was unclear what would happen afterward.
France and Britain both have small contingents of special forces inside Syria, focused like the US troops on battling the Islamic State group.
The US withdrawal will make Russia, which has deployed its air power in support of President Bashar al-Assad, the pre-eminent global power in the conflict.
As Trump tweeted that Russia wanted US forces to stay and blamed the “fake news" media for reporting otherwise, Putin publicly rejoiced over the pullout order.
“The fact that the US has decided to withdraw its troops is right," Putin said during an annual year-end press conference, saying that “on the whole I agree with the US president" on the level of damage inflicted on Islamic State.
Putin, who has described the fall of the Soviet Union as a historic geopolitical disaster, sees Moscow’s longtime ally Syria as a key asset in preserving influence in the Middle East.
Iran’s Shiite clerical regime has also strongly backed Assad, a secular leader from the heterodox Alawite sect.
Turkey opposes Assad and may be emboldened by Trump to attack Kurdish fighters inside Syria, who fought alongside US troops against the Islamic State group.
Turkey links the Kurds who dominate the US-backed Syrian Democratic Forces to a decades-old insurgency at home, but had been reluctant to strike for fear of setting off a crisis if the United States suffered casualties.
Mustefa Bali, a spokesman for the Syrian Democratic Forces, said the fighters would keep up the battle against Islamic State -- but that all bets were off if Turkey attacks.
Bali said the Kurdish forces would keep locked up the Islamic State extremists in their custody -- but alleged that Turkey may target prisons to sow chaos once US troops leave.
French President Emmanuel Macron’s office called the prisoner issue “extremely important for France" and urged discussions to avoid “one of the negative consequences of a hasty retreat."
The Islamic State movement has claimed credit for a slew of bloody attacks around the world, including the 2015 coordinated assault on Paris, and experts estimate that thousands of sympathizers remain.
Meanwhile Germany, which has taken in more than one million refugees stemming in large part from the Syria conflict, questioned Trump’s assessment that the threat was over.
While fighting has largely subsided in Syria and the Islamic State group holds little territory, a political solution remains elusive in ending the war that has killed more than 360,000 and displaced millions since 2011.
The outgoing UN envoy to Syria, Staffan de Mistura, acknowledged Thursday to the Security Council that a goal would not be met of setting up a committee by the end of the year to write a new Syrian constitution.