Nested signal handling leads to excessive slowdown of sig_occurred()

[user202729]

From sagemath/sage#39224

SageMath session:

sage: %%cython
....: from cysignals.signals cimport sig_on, sig_off, sig_occurred
....: from libc.stdlib cimport abort
....: def f():
....:     try:
....:         sig_on()
....:         abort()
....:         sig_off()
....:     except:
....:         raise ValueError
....: def g():
....:     for i in range(50):
....:         ignore = sig_occurred()
sage: %time g()  # fast
CPU times: user 4 μs, sys: 0 ns, total: 4 μs
Wall time: 6.44 μs
sage: f()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
File /tmp/tmpgqxu9lji/spyx/_tmp_tmpfkjflhcs_tmp_f70vs9fj_pyx/_tmp_tmpfkjflhcs_tmp_f70vs9fj_pyx_0.pyx:5, in _tmp_tmpfkjflhcs_tmp_f70vs9fj_pyx_0.f()
      4 try:
----> 5     sig_on()
      6     abort()

RuntimeError: Aborted

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[3], line 1
----> 1 f()

File /tmp/tmpgqxu9lji/spyx/_tmp_tmpfkjflhcs_tmp_f70vs9fj_pyx/_tmp_tmpfkjflhcs_tmp_f70vs9fj_pyx_0.pyx:9, in _tmp_tmpfkjflhcs_tmp_f70vs9fj_pyx_0.f()
      7         sig_off()
      8     except:
----> 9         raise ValueError
     10 def g():
     11     for i in range(50):

ValueError: 
sage: %time g()  # slow
CPU times: user 3.84 s, sys: 133 μs, total: 3.84 s
Wall time: 3.85 s
sage: raise RuntimeError
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[5], line 1
----> 1 raise RuntimeError

RuntimeError: 
sage: %time g()  # fast again
CPU times: user 79.9 ms, sys: 0 ns, total: 79.9 ms
Wall time: 79.9 ms

Note the invocation of g() that is marked # slow.

[user202729]

I guess recursive .__context__ can be accessed. (.__cause__ is the same as .__context__ in the case of raise ValueError from e, but in nested exception then .__cause__ is None but .__context__ is not.)

That said, is there any cleaner way than the current (fragile) method of using reference counting and run the garbage collector? It looks like it will only be a matter of time until someone comes across a case where the garbage collector is run too many times consecutively which leads to unexplained slowdown.

[user202729]

Actually nested signal handling is not needed, you might as well do something more innocuous such as

%%cython
from libc.stdlib cimport abort
from cysignals.signals cimport sig_on
from cysignals.tests import print_sig_occurred
from sage.all import Integer

def f():
    a = [Integer(i) for i in range(100)]
    try:
        sig_on()
        abort()
    except RuntimeError:
        del a

But sig_occurred is introduced for a single purpose in SageMath sagemath/sage#24986 .

[user202729]

How about we throttle the garbage collector to run at most e.g. 10% of the (wall) time? That way, worst case something heavy is done in a place where sig_occurred() must honestly return nonnull and a lot of integers are being destroyed, there's ("only") 10% slowdown. And maybe a user warning.