The US is showing the Saudi-led coalition "how to use intelligence so that you very precisely try to miss killing civilians" while targeting Houthi forces, Mattis told reporters at the Pentagon on Friday.
"We're going to try to make that army, that military of the Saudis, more capable of carrying out what they find to be their military necessity, without killing innocents," he said.
A statement released on Thursday by the United Nations humanitarian coordinator for Yemen, Jamie McGoldrick, noted that Saudi-led air strikes have killed more than 100 civilians in Yemen in the last 10 days.
"This absurd war ... has only resulted in the destruction of the country and the incommensurate suffering of its people, who are being punished as part of a futile military campaign by both sides," he noted.
On December 26 alone, an air strike on a market in Yemen's central Taiz governorate killed 54 people, including eight children, while a separate attack in the western governorate of al-Hudaydah resulted in 14 deaths, according to UN estimates.
"I remind all parties to the conflict, including the Saudi-led coalition, of their obligations under International Humanitarian Law to spare civilians and civilian infrastructure and to always distinguish between civilian and military objects," McGoldrick noted.
Mattis, meanwhile, criticised Houthi forces for storing weapons in residential areas, saying this "doesn't help if they're concerned about civilian casualties".
The US will continue to pursue a diplomatic solution to the war, Mattis told reporters.
Yemen has been decimated by conflict since Houthi rebels, allied with troops loyal to late President Ali Abdullah Saleh, captured large expanses of the country, including Sanaa.
In response, Saudi Arabia launched a massive aerial campaign against the rebels in March 2015, aimed at restoring the government of President Abd-Rabbu Mansour Hadi.
Since then, the Houthis have been dislodged from most of the south, but remain in control of the capital and much of the north.
The country is also facing a deadly cholera outbreak, a direct consequence of the war, that has claimed about 2,000 lives and affected more than one million people since April.