How do I completely reset Mail.app to avoid "Out of Application Memory" errors
(some context, I have been professionally writing mac applications since 1993)
This forum locks your thread after a while so this post is to reply to a previous post since I now know the answer for macOS Monterey 12.3
The issue has not been able to be solved by any of the helpful suggestions on my first post. Nor was it solved by 4 hour-long calls from AppleCare.
Briefly: I upgraded to a M1 iMac 24", Mail starts to crash, the "Out of Application Memory" started to come up. Upgrade was made using Migration Assistant with a TimeMachine backup from a disk.
Annoying UI fact: the Out of Application Memory dialog shows you a list of running apps and how much memory they are using takes a while to come up and the app that is causing the issue (Mail) had already crashed and so was not on this list.
Turns out Mail was inexplicably making copies of all my email in "On My Mac" folders, and that was what was causing it to run out of application ram. They were all in folders called Import-1, Import-2 etc. This copying process seemed to use crazy amounts of ram > 10Gb and each time it did it was gradually filling my drive which meant less space to use for virtual memory and so more crashing.
My original post How do I completely reset Mail.app? - Apple Community which you can see some more of the things I discovred and things I tried that didn't work.
I will post what I did next