HomePod Stere Pairing Issues

Hi, I have recently bought a second HomePod pro and am having huge problems with stereo pairing - it is basically unusable. Pairing works for a while, between an hour and a day - but suddenly only starts playing out of a single speaker. It tends to be after no music has been played for a period.

Sometimes, a restart can fix it - but more often it requires removing the devices any resetting them up - is anyone else having these issues?