Mitigate Python 3.12 Shutdown RuntimeError

[ppigazzini]

Python 3.12 introduced a bug during the interpreter shutdown, see:

• issue gh-104690 Disallow thread creation and fork at interpreter finalization python/cpython#104826
• bugfix [3.12] gh-113964: Don't prevent new threads until all non-daemon threads exit (GH-116677) python/cpython#117029

With Python 3.12.3, this change could potentially be reverted (after testing).

[ppigazzini]

Drafting, python 3.12 intentionally blocked the spawn of a new thread during the shutdown process, I found the official issue upstream (VanRossum involved!):
python/cpython#113964

EDIT: The fix for the RuntimeError issue has already been merged upstream. You can find the relevant pull request here below. With the release of Python 3.12.3, this error should no longer occur.
python/cpython#117029

[ppigazzini]

For reference this is the strack trace in the log on DEV + few workers triggered with:

clear && sudo systemctl restart fishtest@{6543..6545} && sudo journalctl -u fishtest@6543 --since "10 minutes ago"

systemd[1]: Stopping Fishtest Server port 6543...
pserve[17032]: flush
pserve[17032]: .....................done
pserve[17032]: Exception in thread Thread-740:
pserve[17032]: Traceback (most recent call last):
pserve[17032]:   File "/home/fishtest/.pyenv/versions/3.12.2/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
pserve[17032]:     self.run()
pserve[17032]:   File "/home/fishtest/.pyenv/versions/3.12.2/lib/python3.12/threading.py", line 1431, in run
pserve[17032]:     self.function(*self.args, **self.kwargs)
pserve[17032]:   File "/home/fishtest/fishtest/server/fishtest/rundb.py", line 451, in flush_buffers
pserve[17032]:     self.start_timer()
pserve[17032]:   File "/home/fishtest/fishtest/server/fishtest/rundb.py", line 365, in start_timer
pserve[17032]:     self.timer.start()
pserve[17032]:   File "/home/fishtest/.pyenv/versions/3.12.2/lib/python3.12/threading.py", line 992, in start
pserve[17032]:     _start_new_thread(self._bootstrap, ())
pserve[17032]: RuntimeError: can't create new thread at interpreter shutdown
systemd[1]: Stopped Fishtest Server port 6543.