Deadlock: TakeData.SetCurrentTake() from Python holds the GIL

nik684

Hello. I found some strange issues with C4D freezing when developing a tool to manage Takes.
I tried to debug it with Claude and here is the report:

Product: Cinema 4D 2027.0.0 (Python 3.11.4), Windows 11 Pro. Redshift is the active renderer, but the repro does not depend on it.

Summary

c4d.modules.takesystem.TakeData.SetCurrentTake() called from Python on the main thread does not release the GIL. While the take switch runs, Cinema 4D waits for a viewport redraw (or scene evaluation) running in another thread. If that thread evaluates Python (a Python tag, an XPresso Python node, or any other Python-based scene element), it waits for the GIL held by the main thread. Cinema 4D stops responding and never recovers; the process has to be killed.

The same switch made from C++ with the GIL released around the call never hangs. This suggests the Python wrapper should release the GIL for the duration of the call (Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS), as c4d.CallCommand apparently already does.

Steps to reproduce

Start Cinema 4D 2027 with the default layout (one perspective view is enough).
Run the attached take_switch_gil_deadlock_repro.py from the Script Manager. The script:
- creates a new document with one cube that carries a Python tag, whose main() takes about 20 ms (standing for a heavy rig);
- adds one child Take that overrides the cube's position;
- opens an async GeDialog whose Timer() switches between the Main Take and the child Take every 50 ms with TakeData.SetCurrentTake() followed by c4d.EventAdd().

Expected: the dialog keeps switching Takes.

Actual: Cinema 4D stops responding at the first or one of the first switches and stays frozen. In our test it hung at the very first switch; py-spy shows the main thread at td.SetCurrentTake(target).

With a Python tag whose main() is just pass, the window is very small and 1000 switches usually pass; the 20 ms tag only makes the race reproducible. In production we hit it with a character rig whose Python tag lives inside an XRef, by clicking in a Python dialog that switches Takes.

Stacks of the hung process (py-spy dump --native)

Main thread, holding the GIL, waiting inside the take switch:

Thread 29216 (idle): "MainThread"
    ZwWaitForAlertByThreadId (ntdll.dll)
    RtlSleepConditionVariableCS (ntdll.dll)
    SleepConditionVariableCS (KERNELBASE.dll)
    (Cinema 4D.exe)
    (Cinema 4D.exe)
    (c4d_base.xdl64)
    (c4d_base.xdl64)
    (c4d_base.xdl64)
    (c4d_manager.xdl64)
    (python.xdl64)                      <- TakeData.SetCurrentTake wrapper
    PyType_GenericNew (python311.dll)
    PyObject_Vectorcall (python311.dll)
    PyEval_EvalFrameDefault (python311.dll)
    Timer (take_switch_gil_deadlock_repro.py:69)   <- td.SetCurrentTake(target)

Redraw / evaluation thread, inside the Python tag, waiting for the GIL:

Thread 28012 (idle)
    NtWaitForSingleObject (ntdll.dll)
    WaitForSingleObjectEx (KERNELBASE.dll)
    PyObject_Call_Prepend (python311.dll)
    PyEval_RestoreThread (python311.dll)      <- waits for the GIL
    ...
    PyObject_CallFunction (python311.dll)
    (python.xdl64)
    (c4d_base.xdl64)
    (Cinema 4D.exe)
    main (Python tag, line 5)

In production the main thread variant was SetCurrentTake -> c4d_manager -> c4d_base -> hair.xdl64 -> ca.xdl64 -> c4d_base -> SwitchToThread (a spin wait). The other thread was a job thread under ca.xdl64 in PyGILState_Ensure, about to run the rig's Python tag. In the reproducible cases CHECKISRUNNING_VIEWDRAWING was 1 at the time of the switch.

What we measured

Switching Takes from Python every 50 ms with EventAdd() between the switches, in a scene with a rig whose Python tag lives inside an XRef: hang at the 13th switch.
The same loop, but with the switch made by a C++ CommandData called through c4d.CallCommand(), which also releases the GIL (PyEval_SaveThread / PyEval_RestoreThread) around TakeData::SetCurrentTake(): 1000 switches, 780 of them while a redraw was running, no hang.
300 switches in a single Python call without EventAdd(), so no redraw can start in between: no hang. The deadlock needs a redraw that is already running when the switch starts.
c4d.StopAllThreads() does not stop a running viewport redraw: it returns immediately and CHECKISRUNNING_VIEWDRAWING stays 1.
c4d.EventAdd() does not start a redraw while the main thread is still in Python. A redraw started between two Python calls (e.g. two Timer() ticks) keeps running into the next one.
An undoable parameter edit in the same setup (StartUndo / AddUndo(UNDOTYPE_CHANGE) / parameter change / EndUndo every 50 ms): 1000 edits, no hang.
We did, however, observe the same kind of hang in production in two more calls, both while a viewport redraw was evaluating a Python tag. We could not reproduce them in a minimal scene.
- BaseDocument.AddUndo(): main thread in AddUndo -> hair -> ca -> SwitchToThread.
- maxon.GraphTransaction.Commit(): main thread on a condition variable, a drawport thread in PyGILState_Ensure.

Workarounds we use

Before a scene write from Python, wait until c4d.CheckIsRunning(c4d.CHECKISRUNNING_VIEWDRAWING) has stayed 0 for about 10 ms, sleeping in between (time.sleep releases the GIL so the redraw can finish). One redraw runs in parts with the flag off for a moment between them, so a single check is not enough. This narrows the window but cannot close it.
Make the Take switch from a small C++ command plugin that releases the GIL around TakeData::SetCurrentTake(). Python passes the Take through the world container and calls c4d.CallCommand(). This removes the deadlock, but every Python tool would need such a native helper.

Questions / request

Could the Python wrappers of TakeData.SetCurrentTake() release the GIL while the native call runs? The same applies to other calls that can wait for scene evaluation or redraws, e.g. BaseDocument.AddUndo(), GraphTransaction.Commit() and BaseDocument.ExecutePasses().
Is there a supported way for a Python plugin to block or pause viewport redraws for the duration of a set of scene changes? Or to wait until no redraw or scene evaluation is running, something stronger than polling CHECKISRUNNING_VIEWDRAWING?
Is calling TakeData.SetCurrentTake() from a GeDialog.Timer() or a GeUserArea mouse handler supported, or should it be deferred to some other context?

take_switch_gil_deadlock_repro.py

ferdinand

Hello @nik684,

Welcome to the Maxon developers forum and its community, it is great to have you with us!

Getting Started

Before creating your next postings, we would recommend making yourself accustomed with our forum and support procedures. You did not do anything wrong, we point all new users to these rules.

Forum Overview: Provides a broad overview of the fundamental structure and rules of this forum, such as the purpose of the different sub-forums or the fact that we will ban users who engage in hate speech or harassment.
Support Procedures: Provides a more in detail overview of how we provide technical support for APIs here. This topic will tell you how to ask good questions and limits of our technical support.
Forum Features: Provides an overview of the technical features of this forum, such as Markdown markup or file uploads.

It is strongly recommended to read the first two topics carefully, especially the section Support Procedures: How to Ask Questions.

About your First Question

This has to be one of the most thorough bug reports I have ever seen. Well done! Maybe tell your agent next time to make it a bit shorter, I have a not too long attention span

Could the Python wrappers of TakeData.SetCurrentTake() release the GIL while the native call runs ?

You were right, CPyTakeData::SetCurrentTake was missing the GIL release and acquire around its takeData->SetCurrentTake(take) call as it is customary for methods that write something. I have just pushed a build to the build server which changes that. I will then try to run your script there once I have that build. And even when the fix does not resolve your issue, I will submit it anyway, as the GIL should be released by convention around that call I would say.

Is there a supported way for a Python plugin to block or pause viewport redraws for the duration of a set of scene changes?

No, there is unfortunately not. You are in a more or less curated threading environment in Python. The Python VM calls your code at specific points in time and you are more or less at its mercy. There are c4d.StopAllThreads and c4d.threading.GeThreadLock but using them is rarely a good idea. And some drawing threads might even ignore them.

Is calling TakeData.SetCurrentTake() from a GeDialog.Timer() or a GeUserArea mouse handler supported, or should it be deferred to some other context?

That is indeed the main question. You can look at the all new Scene Excecution manual which I wrote for 2027 to get a better understanding of this. In short, changing the active take of a loaded scene is a scene modification and therefore must be main thread bound. Timer and Mouse/Keyboard IO functions usually run on the MT, but I would add an explicit check to such functions just to me sure. So, on paper, this is fine.

Where I can see a bit an issue is with your INTERVAL_MS in the demo script you gave us, it is set to 50ms. Quoting the warning I once added to GetTimer:

"Depending on the speed of the computer, the operating system, the complexity of the dialog and the threads running in the background, there is no guarantee that event messages will occur on a regular basis. Using a value of 500 ms should be no problem but if using a value of 1 ms one might get events with the following time spaces: 3 ms, 76 ms, 15 ms, 19 ms, 67 ms…"

With 50ms you are dangerously close to saturating the event system. Although it might not look like it, SetCurrentTake is a VERY expensive call. Doing this every 50ms with all Python shenanigans in between is not great. You would be better served with a value such as 100 or even 250ms.

I am like 66% confident that the GIL release I added will fix your issue. But all that event and timer stuff could also throw a wrench into things. But I will cross that bridge once I have a build of my fix, and your code still freezes, as that is probably not fun to debug. You should take as a lesson here that writing takes is not cheap. It can entail a lot of data copying and scene traversal to bring the current scene into its new state.

I will post here an update once I have a final answer.

Cheers,
Ferdinand

ferdinand

Okay, this did fix it. I made a merge request. It will not be included in the next release (which will arrive very soon). I hope to get it into the 2027 Q1 release (which is also already on the finish line), otherwise it will come with Q2. I also fixed some other methods in take take system, as quite a few were missing proper GIL handling.

nik684

Wow, first of all I'm impressed by quality and speed of this support! I'm newbie here so it was a pleasant surprise. As a maxscript guy I'm not used to this
I'll study every link you posted and for sure will tweak the agent to be more concise in future.
Thanks a lot for the upcoming fix.

Regarding test script. I guess that script was written intentionally stressfull by Claude to reproduce the issue. Ofc I'm not spamming Take switch that frequently in the production tool, but I got freezes even with pretty calm usual switches. Now we know why.

Taking advantage of the moment, I would like to mention that Override Groups are pretty greedy in terms of memory for some reason. When I put a lot of objects in Override Group Cinema takes much more RAM than when I'm using usual parameter overrides with the same amounf of objects/parameters. It doesn't look like a bug, but maybe there is some room for improvement. Should I create another post with this issue?

nik684

Also worth to note:
We hit the same freeze in BaseDocument.AddUndo() and maxon.GraphTransaction.Commit() on the main thread while a Python tag was drawing.
py-spy showed the same pattern: the main thread holds the GIL and waits, and the draw/ca thread sits in PyGILState_Ensure.
Could those wrappers be missing the GIL release too?

Undo maybe be related to Take switch though.

ferdinand

You picked a terrible spot to start your Cinema journey, the take system while beloved by users, is both complicated and not performance-wise not great when it comes to the API

That is why I added my "It can entail a lot of data copying and scene traversal to bring the current scene into its new state" warning above. The take system entails a lot of copying of data containers or at least parts of them. The data itself is probably only a small portion of the spike you see, what makes this then really expensive, is the fact that when you make half your scene dirty by applying new parameters to it, you also have to rebuild half your scene from scratch which eats a lot of memory.

But I cannot really do anything about this, as this is not my domain, I do not own the take system, just the Python bindings to it. When you consider the take system too slow, I would encourage you to make a user report in our Support Center. This is also probably not something you fix in five minutes.

Cheers,
Ferdinand

ferdinand

@nik684 said in Deadlock: TakeData.SetCurrentTake() from Python holds the GIL:

Also worth to note:
We hit the same freeze in BaseDocument.AddUndo() and maxon.GraphTransaction.Commit() on the main thread while a Python tag was drawing.
py-spy showed the same pattern: the main thread holds the GIL and waits, and the draw/ca thread sits in PyGILState_Ensure.
Could those wrappers be missing the GIL release too?

Undo maybe be related to Take switch though.

AddUndo just as some other Undo methods does not release the GIL, I just checked. But other than for the take system, this could be intentional here, due to adding an undo could imply changes to PyObject instances (which are not allowed while you drop the GIL).

For maxon.GraphTransaction.Commit() I would have to check, things work entirely different there, the Maxon API is not implemented in a traditional way, things work much differently there. I would need code and a context for the latter. For the former I probably need a good night sleep to think about it and a reason where this would be relevant.

Cheers,
Ferdinand