Find My is extremely slow on MacOS when looking at "People" tab

A very strange problem has been plaguing my MBP (16" Nov 2023) running Tahoe (and before in Sequoia).

When I launch Find My (with the default tab, selected for "People"), it takes nearly a minute of spinning beach ball before the map loads in on my location. When I pick one of the persons on my list of people to show their location, it takes another 30 seconds, and the application is excruciatingly slow. Switching between different people on the list always results in taking forever (nearly a minute) of spinning beach ball before the map loads and location shows. If I switch to "Devices", map opens instantaneously, and location of any of the people's devices is retrieved within a few seconds. In other words, the application behaves as expected. On all my other Macs and devices (iPad, iPhone), all works as expected.

When I log into this same Mac as a different user, Find My behaves as expected, and doesn't show any performance issues. They can all easily switch between people and the map snaps to their location quite quickly (as expected). Only on my own profile, and only on the "People" does it exhibit this very annoying issue. I don't have any "Items" to test this with the third tab.

I tried deleting preferences, caches, and various other system files related to Find Me, but it didn't help.

Does anyone have any additional ideas, how to resolve this bizarre problem? I don't want to have to completely delete my profile and re-create it, restoring data from backup. There must be some logical answer.