from your tellings the following article / behaviour comes in my mind:
If you’re using IMAP IDLE and have a lot of clients connected, with watches on various folders, it’s possibel that you exceed the common default limit of 32768 directory watches. You should be able to simply increase the limit as described in the linked guide.