Can't turn on Handwashing Reminders

When I try to turn on the Handwashing Reminders, the button isn't clickable and appears bellow that "Reminders aren't available because you haven't set a home address in your My Card. You can add your address in Contacts.". The problem is that I do have a home location in my card and I already tried deleting it and adding it again. I also restarted both my iPhone (16 Pro) and Apple Watch (Series 10). Both are with the latest software update. Still doesn't work.

Also the address on my card is correct, with the map and the red pin displayed beside it.