Posts made by nik684

nik684

Also worth to note:
We hit the same freeze in BaseDocument.AddUndo() and maxon.GraphTransaction.Commit() on the main thread while a Python tag was drawing.
py-spy showed the same pattern: the main thread holds the GIL and waits, and the draw/ca thread sits in PyGILState_Ensure.
Could those wrappers be missing the GIL release too?

Undo maybe be related to Take switch though.

nik684

Wow, first of all I'm impressed by quality and speed of this support! I'm newbie here so it was a pleasant surprise. As a maxscript guy I'm not used to this
I'll study every link you posted and for sure will tweak the agent to be more concise in future.
Thanks a lot for the upcoming fix.

Regarding test script. I guess that script was written intentionally stressfull by Claude to reproduce the issue. Ofc I'm not spamming Take switch that frequently in the production tool, but I got freezes even with pretty calm usual switches. Now we know why.

Taking advantage of the moment, I would like to mention that Override Groups are pretty greedy in terms of memory for some reason. When I put a lot of objects in Override Group Cinema takes much more RAM than when I'm using usual parameter overrides with the same amounf of objects/parameters. It doesn't look like a bug, but maybe there is some room for improvement. Should I create another post with this issue?

nik684

Hello. I found some strange issues with C4D freezing when developing a tool to manage Takes.
I tried to debug it with Claude and here is the report:

Product: Cinema 4D 2027.0.0 (Python 3.11.4), Windows 11 Pro. Redshift is the active renderer, but the repro does not depend on it.

Summary

c4d.modules.takesystem.TakeData.SetCurrentTake() called from Python on the main thread does not release the GIL. While the take switch runs, Cinema 4D waits for a viewport redraw (or scene evaluation) running in another thread. If that thread evaluates Python (a Python tag, an XPresso Python node, or any other Python-based scene element), it waits for the GIL held by the main thread. Cinema 4D stops responding and never recovers; the process has to be killed.

The same switch made from C++ with the GIL released around the call never hangs. This suggests the Python wrapper should release the GIL for the duration of the call (Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS), as c4d.CallCommand apparently already does.

Steps to reproduce

Start Cinema 4D 2027 with the default layout (one perspective view is enough).
Run the attached take_switch_gil_deadlock_repro.py from the Script Manager. The script:
- creates a new document with one cube that carries a Python tag, whose main() takes about 20 ms (standing for a heavy rig);
- adds one child Take that overrides the cube's position;
- opens an async GeDialog whose Timer() switches between the Main Take and the child Take every 50 ms with TakeData.SetCurrentTake() followed by c4d.EventAdd().

Expected: the dialog keeps switching Takes.

Actual: Cinema 4D stops responding at the first or one of the first switches and stays frozen. In our test it hung at the very first switch; py-spy shows the main thread at td.SetCurrentTake(target).

With a Python tag whose main() is just pass, the window is very small and 1000 switches usually pass; the 20 ms tag only makes the race reproducible. In production we hit it with a character rig whose Python tag lives inside an XRef, by clicking in a Python dialog that switches Takes.

Stacks of the hung process (py-spy dump --native)

Main thread, holding the GIL, waiting inside the take switch:

Thread 29216 (idle): "MainThread"
    ZwWaitForAlertByThreadId (ntdll.dll)
    RtlSleepConditionVariableCS (ntdll.dll)
    SleepConditionVariableCS (KERNELBASE.dll)
    (Cinema 4D.exe)
    (Cinema 4D.exe)
    (c4d_base.xdl64)
    (c4d_base.xdl64)
    (c4d_base.xdl64)
    (c4d_manager.xdl64)
    (python.xdl64)                      <- TakeData.SetCurrentTake wrapper
    PyType_GenericNew (python311.dll)
    PyObject_Vectorcall (python311.dll)
    PyEval_EvalFrameDefault (python311.dll)
    Timer (take_switch_gil_deadlock_repro.py:69)   <- td.SetCurrentTake(target)

Redraw / evaluation thread, inside the Python tag, waiting for the GIL:

Thread 28012 (idle)
    NtWaitForSingleObject (ntdll.dll)
    WaitForSingleObjectEx (KERNELBASE.dll)
    PyObject_Call_Prepend (python311.dll)
    PyEval_RestoreThread (python311.dll)      <- waits for the GIL
    ...
    PyObject_CallFunction (python311.dll)
    (python.xdl64)
    (c4d_base.xdl64)
    (Cinema 4D.exe)
    main (Python tag, line 5)

In production the main thread variant was SetCurrentTake -> c4d_manager -> c4d_base -> hair.xdl64 -> ca.xdl64 -> c4d_base -> SwitchToThread (a spin wait). The other thread was a job thread under ca.xdl64 in PyGILState_Ensure, about to run the rig's Python tag. In the reproducible cases CHECKISRUNNING_VIEWDRAWING was 1 at the time of the switch.

What we measured

Switching Takes from Python every 50 ms with EventAdd() between the switches, in a scene with a rig whose Python tag lives inside an XRef: hang at the 13th switch.
The same loop, but with the switch made by a C++ CommandData called through c4d.CallCommand(), which also releases the GIL (PyEval_SaveThread / PyEval_RestoreThread) around TakeData::SetCurrentTake(): 1000 switches, 780 of them while a redraw was running, no hang.
300 switches in a single Python call without EventAdd(), so no redraw can start in between: no hang. The deadlock needs a redraw that is already running when the switch starts.
c4d.StopAllThreads() does not stop a running viewport redraw: it returns immediately and CHECKISRUNNING_VIEWDRAWING stays 1.
c4d.EventAdd() does not start a redraw while the main thread is still in Python. A redraw started between two Python calls (e.g. two Timer() ticks) keeps running into the next one.
An undoable parameter edit in the same setup (StartUndo / AddUndo(UNDOTYPE_CHANGE) / parameter change / EndUndo every 50 ms): 1000 edits, no hang.
We did, however, observe the same kind of hang in production in two more calls, both while a viewport redraw was evaluating a Python tag. We could not reproduce them in a minimal scene.
- BaseDocument.AddUndo(): main thread in AddUndo -> hair -> ca -> SwitchToThread.
- maxon.GraphTransaction.Commit(): main thread on a condition variable, a drawport thread in PyGILState_Ensure.

Workarounds we use

Before a scene write from Python, wait until c4d.CheckIsRunning(c4d.CHECKISRUNNING_VIEWDRAWING) has stayed 0 for about 10 ms, sleeping in between (time.sleep releases the GIL so the redraw can finish). One redraw runs in parts with the flag off for a moment between them, so a single check is not enough. This narrows the window but cannot close it.
Make the Take switch from a small C++ command plugin that releases the GIL around TakeData::SetCurrentTake(). Python passes the Take through the world container and calls c4d.CallCommand(). This removes the deadlock, but every Python tool would need such a native helper.

Questions / request

Could the Python wrappers of TakeData.SetCurrentTake() release the GIL while the native call runs? The same applies to other calls that can wait for scene evaluation or redraws, e.g. BaseDocument.AddUndo(), GraphTransaction.Commit() and BaseDocument.ExecutePasses().
Is there a supported way for a Python plugin to block or pause viewport redraws for the duration of a set of scene changes? Or to wait until no redraw or scene evaluation is running, something stronger than polling CHECKISRUNNING_VIEWDRAWING?
Is calling TakeData.SetCurrentTake() from a GeDialog.Timer() or a GeUserArea mouse handler supported, or should it be deferred to some other context?

take_switch_gil_deadlock_repro.py