Griffiths' visit comes as the ceasefire in Hodeida was generally holding.
A pro-government official said that the loyalists were surprised by reports of a port handover.
The team led by Patrick Cammaert, a retired Dutch general, made their first stop in Aden where the internationally-recognised government is based.
The agreement also included a planned prisoner swap involving about 15,000 detainees.
The conflict has also pushed 14 million people to the brink of famine.
The Iranian-aligned Houthi group and the Saudi-backed government of Abd-Rabbu Mansour Hadi agreed to cease fighting in the Red Sea city and withdraw forces.
At least 29 fighters, including 22 Huthi rebels, have been killed in clashes and air strikes in Hodeida province.
Analysts warned the ceasefire accord's success depends on sustained international pressure.
Both Yemeni parties have threatened to leave the talks if certain demands are not met.
UN aid chief Mark Lowcock said conditions had deteriorated alarmingly since his last visit to Yemen.
More than 400 combatants have been killed in 10 days of clashes in Hodeida.
Medics at hospitals inside the city reported 47 rebels had been killed in overnight ground fighting and air raids by a Saudi-led coalition supporting the government.
53 Huthi rebels have been killed and dozens wounded in battles and air strikes over the past 24 hours.
A total of 32 insurgents were killed and 14 others wounded in clashes and air strikes since Saturday.
Nearly 10,000 people have been killed in Yemen's conflict since 2015, when the Saudi-led alliance intervened in support of the government.
Rebel leader Abdulmalik al-Huthi called for reinforcements to repel the advance of the UAE-backed government forces, after ongoing fighting left nearly 350 people dead.
The Yemeni government and its allies launched their offensive on Wednesday. At least 139 combatants have already been killed, according to sources.
The rebels launched the attack on the coast road from the government-held ports of Khokha and Mokha along which Saudi-backed troops advanced on Hodeida earlier this week, the military source said.
The assessment by humanitarian and port officials comes as Saudi Arabia and its regional allies push into the fourth year of their military intervention in Yemen.