How can I get the SUM formula to output “0h 0m” or “0h” instead of just “0”

If all the cells included in the SUM are empty, the sum will be the number 0 and you cannot format it as a duration. But you can include DURATION(0) in the SUM, or added to it, to force it to be a duration. Format the cell to show it as 0h. Or you can use DURATION(,,0) and use automatic formatting.

Examples:

=SUM(B6:O6,DURATION(0))

or

=DURATION(0)+SUM(B6:O6)

Assuming the cell value is something that Numbers can interpret as a duration, use the Cell Format inspector:

Set the Cell Format to Duration, then under Custom Units you can choose which units (weeks, days, hours, minutes, seconds, milliseconds) to display.

Not sure I follow the screenshots, but 0 indicates the cell contains a Number value while “0h 0m” and “0h” indicate that the cell contains a Duration value.

Durations (spans of time as opposed to a point in time) result from subtracting one Date & Time from another or by adding or subtracting a number (which Numbers interprets as days) to or from a Date & Time. Adding a Duration to a Duration also results in a Duration.

Durations can be input using w d h m ms shorthand.

Documentation on Number vs Duration data formats is terse, but worth a read:

Format dates, currency, and more in Numbers on Mac - Apple Support

SG