Global Moderators group

ferdinand

your question is a bit ambiguous. What is 'Connector Debug output' for you and what is exactly 'inside VSCode' for you, especially under the condition of 'limited access to the machine'?

The connection between VSCode and Cinema 4D to exchange scripts and console output of Cinema 4D is proprietary. It is an websocket connection that runs on the port you define in Cinema (by default 7788), e.g., 127.0.0.1:7788. The payload is JSON but I would rather not publicly document its details, primarily because I do not want to give the impression that there is any guarantee that the protocol is meant to be a stable API third parties can hook into. But since this is a VSCode extension, you can of course just look at the source to understand the details, or ask an agent to do it for you.

When you mean with debug output the actual debugger, then we are using the Microsoft library debugpy. We use port 5678 but you could also use any other port because your app is the server, not the consumer. I.e., the debug IP could be 127.0.0.1:5678. The old (2026.3) docs contained a section that showed how to manually attach from your code to a debugpy client, e.g. VS Code. But I have removed this section in the 2027 Debugging Manual because the debugger is currently not properly working under MacOS and I did not find the time to fix that for 202.0.0 (but I will fix it within 2027.X.X).

Note that the debugger connection is made from your code; your code is the server, VSCode is the client. debugpy is a proprietary Microsoft library and the only meaningful enterprise debugger for Python at the moment. There is the builtin pdb library which you can use but it IMHO has its limits. There are also some smaller third party alternatives, but they are all inconsequential and I do not know if they work with Cinema 4D. debugpy is using the open DAP protocol, so it is not impossible to create your own consumer, but it is probably also not trivial (I am mentioning this because your message somewhat implies that you want to sidestep the VS Code debugger interface).

Cheers,
Ferdinand

ferdinand

@nik684 said in Deadlock: TakeData.SetCurrentTake() from Python holds the GIL:

Also worth to note:
We hit the same freeze in BaseDocument.AddUndo() and maxon.GraphTransaction.Commit() on the main thread while a Python tag was drawing.
py-spy showed the same pattern: the main thread holds the GIL and waits, and the draw/ca thread sits in PyGILState_Ensure.
Could those wrappers be missing the GIL release too?

Undo maybe be related to Take switch though.

AddUndo just as some other Undo methods does not release the GIL, I just checked. But other than for the take system, this could be intentional here, due to adding an undo could imply changes to PyObject instances (which are not allowed while you drop the GIL).

For maxon.GraphTransaction.Commit() I would have to check, things work entirely different there, the Maxon API is not implemented in a traditional way, things work much differently there. I would need code and a context for the latter. For the former I probably need a good night sleep to think about it and a reason where this would be relevant.

Cheers,
Ferdinand

ferdinand

You picked a terrible spot to start your Cinema journey, the take system while beloved by users, is both complicated and not performance-wise not great when it comes to the API

That is why I added my "It can entail a lot of data copying and scene traversal to bring the current scene into its new state" warning above. The take system entails a lot of copying of data containers or at least parts of them. The data itself is probably only a small portion of the spike you see, what makes this then really expensive, is the fact that when you make half your scene dirty by applying new parameters to it, you also have to rebuild half your scene from scratch which eats a lot of memory.

But I cannot really do anything about this, as this is not my domain, I do not own the take system, just the Python bindings to it. When you consider the take system too slow, I would encourage you to make a user report in our Support Center. This is also probably not something you fix in five minutes.

Cheers,
Ferdinand

ferdinand

Okay, this did fix it. I made a merge request. It will not be included in the next release (which will arrive very soon). I hope to get it into the 2027 Q1 release (which is also already on the finish line), otherwise it will come with Q2. I also fixed some other methods in take take system, as quite a few were missing proper GIL handling.

ferdinand

Hello @nik684,

Welcome to the Maxon developers forum and its community, it is great to have you with us!

Getting Started

Before creating your next postings, we would recommend making yourself accustomed with our forum and support procedures. You did not do anything wrong, we point all new users to these rules.

Forum Overview: Provides a broad overview of the fundamental structure and rules of this forum, such as the purpose of the different sub-forums or the fact that we will ban users who engage in hate speech or harassment.
Support Procedures: Provides a more in detail overview of how we provide technical support for APIs here. This topic will tell you how to ask good questions and limits of our technical support.
Forum Features: Provides an overview of the technical features of this forum, such as Markdown markup or file uploads.

It is strongly recommended to read the first two topics carefully, especially the section Support Procedures: How to Ask Questions.

About your First Question

This has to be one of the most thorough bug reports I have ever seen. Well done! Maybe tell your agent next time to make it a bit shorter, I have a not too long attention span

Could the Python wrappers of TakeData.SetCurrentTake() release the GIL while the native call runs ?

You were right, CPyTakeData::SetCurrentTake was missing the GIL release and acquire around its takeData->SetCurrentTake(take) call as it is customary for methods that write something. I have just pushed a build to the build server which changes that. I will then try to run your script there once I have that build. And even when the fix does not resolve your issue, I will submit it anyway, as the GIL should be released by convention around that call I would say.

Is there a supported way for a Python plugin to block or pause viewport redraws for the duration of a set of scene changes?

No, there is unfortunately not. You are in a more or less curated threading environment in Python. The Python VM calls your code at specific points in time and you are more or less at its mercy. There are c4d.StopAllThreads and c4d.threading.GeThreadLock but using them is rarely a good idea. And some drawing threads might even ignore them.

Is calling TakeData.SetCurrentTake() from a GeDialog.Timer() or a GeUserArea mouse handler supported, or should it be deferred to some other context?

That is indeed the main question. You can look at the all new Scene Excecution manual which I wrote for 2027 to get a better understanding of this. In short, changing the active take of a loaded scene is a scene modification and therefore must be main thread bound. Timer and Mouse/Keyboard IO functions usually run on the MT, but I would add an explicit check to such functions just to me sure. So, on paper, this is fine.

Where I can see a bit an issue is with your INTERVAL_MS in the demo script you gave us, it is set to 50ms. Quoting the warning I once added to GetTimer:

"Depending on the speed of the computer, the operating system, the complexity of the dialog and the threads running in the background, there is no guarantee that event messages will occur on a regular basis. Using a value of 500 ms should be no problem but if using a value of 1 ms one might get events with the following time spaces: 3 ms, 76 ms, 15 ms, 19 ms, 67 ms…"

With 50ms you are dangerously close to saturating the event system. Although it might not look like it, SetCurrentTake is a VERY expensive call. Doing this every 50ms with all Python shenanigans in between is not great. You would be better served with a value such as 100 or even 250ms.

I am like 66% confident that the GIL release I added will fix your issue. But all that event and timer stuff could also throw a wrench into things. But I will cross that bridge once I have a build of my fix, and your code still freezes, as that is probably not fun to debug. You should take as a lesson here that writing takes is not cheap. It can entail a lot of data copying and scene traversal to bring the current scene into its new state.

I will post here an update once I have a final answer.

Cheers,
Ferdinand

ferdinand

Hey @Marite,

Welcome to the Maxon developers forum and its community, it is great to have you with us!

Getting Started

Before creating your next postings, we would recommend making yourself accustomed with our forum and support procedures. You did not do anything wrong, we point all new users to these rules.

Forum Overview: Provides a broad overview of the fundamental structure and rules of this forum, such as the purpose of the different sub-forums or the fact that we will ban users who engage in hate speech or harassment.
Support Procedures: Provides a more in detail overview of how we provide technical support for APIs here. This topic will tell you how to ask good questions and limits of our technical support.
Forum Features: Provides an overview of the technical features of this forum, such as Markdown markup or file uploads.

It is strongly recommended to read the first two topics carefully, especially the section Support Procedures: How to Ask Questions.

About your First Posting

Thank you for sharing your project! And please excuse the long wait time in the posting queue, I somehow managed to overlook your posting.

Cheers,
Ferdinand

ferdinand

Hey,

please excuse the long delay. Please have a look at Support Procedures for future topics, we do not allow for multi-question topics, as they have a tendency to derail.

Is there a supported way for a C++ plugin to continuously receive ordered mouse movement events during a keyboard-initiated modal operation, without any mouse button held down?

Yes, there are multiple ways to do this. I cannot highlight each way in all details due to the broad nature of this question and topic. Some ways are:

Implement SceneHookData and implement ::MouseInput, ::KeyboardInput etc. Scene hooks are sort of spider in the net node types which exist always exactly once in each document. They are often used to do command and control tasks. Things like render regions or the simulation system in Cinema are all backed by scene hooks.
Implement a GeDialog, start a timer via GeDialog::SetTimer and on each tick, poll the current input state yourself via GetInputState (the current state) or GetInputEvent (the last enqueued input event, could lie in the past).
Implement a thread or job, and poll from there on each tick.

Can EditorWindow::BfGetInputEvent() be used for this purpose?

EditorWindow::BfGetInputEvent() just like GeDialog::GetInputEvent and similar methods is a specialization of the generic cinema::GetInputEvent. It is often better to use the global functions rather than the specific ones, unless one wants to filter out all events that do not go to a specific viewport or dialog. As hinted at above, you often also want to poll for the state and not for events, as they can be consumed (the user is still holding down "X" but something has already consumed this event) and temporally offset (the user is not pressing x anymore, but the event is still last in queue).

Does calling BfGetInputEvent() remove events from the queue ...

I would quite frankly have to check this, and this is probably not an easy "look it up in file foo.c on line 4555". Without checking, I would say that the InputEvent methods do not consume the events, as there is no way to signal back that you have consumed an event. The canonical way to consume events are plugin methods such as the scene hook methods discussed above, or similar methods of other plugin types. There you can then return true ore false to indicate if you want to consume an event or not.

If BfGetInputEvent() is not suitable, which API or callback mechanism would Maxon recommend for this type of keyboard-initiated continuous viewport interaction? Are there specific thread, window ownership, or lifecycle restrictions?

This is exactly why we disallow multi question topics I could write an essay alone on this because while this is a precise question it covers so much ground. In general, you should avoid polling the input state off main thread if you can. The input methods are not explicitly thread-unsafe but I would still avoid it. The reason is that the plugin type specific methods which are about inputs, such as SceneHookData::MouseInput, ObjectData::MoveHandle, or ToolData::KeyboardInput usually run on the MT. I would advise the against polling inputs in heavily parallelized methods such as SceneHookData::Execute, ObjectData::GetVirtualsObjects, TagData::Execute. While it might not provoke crashes, it is for sure not good for performance. Polling for the input in your own custom thread spawned from the MT is fine.

But you usually do not have to use threading, there should be a plugin interface with input methods which fits your needs. And when the plugin interface you use does not have an input polling function, e.g., you implement a tag, it means that you should not poll for inputs there and use another plugin type or rethink your whole design. You could for sure make it technically work that a tag behaves differently depending on the key(s) the user presses, but from a Maxon point of view, this would be a rogue UX.

Cheers,
Ferdinand

ferdinand

Hey,

sorry for the delay. I checked and it is not possible to have arbitrary node systems, i.e., node systems that are neither scene nor material nodes. In the backed of things, you actually have almost all you need, so you could implement a node system that outputs sound files for example. One of the major hurdles is that this menu is hard-coded at the moment:

So, there is no way to display your node system of a custom node space purpose type.

Cheers,
Ferdinand

ferdinand

Hello @davidz,

Welcome to the Maxon developers forum and its community, it is great to have you with us!

Getting Started

Before creating your next postings, we would recommend making yourself accustomed with our forum and support procedures. You did not do anything wrong, we point all new users to these rules.

Forum Overview: Provides a broad overview of the fundamental structure and rules of this forum, such as the purpose of the different sub-forums or the fact that we will ban users who engage in hate speech or harassment.
Support Procedures: Provides a more in detail overview of how we provide technical support for APIs here. This topic will tell you how to ask good questions and limits of our technical support.
Forum Features: Provides an overview of the technical features of this forum, such as Markdown markup or file uploads.

It is strongly recommended to read the first two topics carefully, especially the section Support Procedures: How to Ask Questions.

About your First Question

Please excuse the long delay. Please also have a look at our Support Procedures: How to Ask Questions, topics should not contain multiple questions as this tends to produce chaotic topics.

Thread Safety & UI Blocking: How are you structuring long-running operations (like heavy geometry processing or batch asset syncs) to ensure C4D’s UI remains fluid?

Cinema 4D does not offer true parallelism in its API in the sense of Python's multiprocessing, only simple de-blocking threading. I.e., you cannot make things faster, you can only make them not block things. This can be achieved via c4d.threading, specifically C4DThread.

Examples:
https://developers.maxon.net/docs/py/2027_0_0/search.html?q=C4DThread
https://developers.maxon.net/forum/topic/14395/c4d-threading-in-python-doesn-t-work-as-expect/5

Module Live-Reloading: When building complex multi-file plugins, what is your preferred approach for hot-reloading modified code without constantly restarting C4D?

You can use mxutils.LocalImportPath, it has a builtin module reload feature within the bounds of trivial import processes, i.e., imports that do not have to happen in a certain order. When you have:

with LocalImportPath(__file__, autoReload=IS_DEBUG):
    import a
    import b
    import c

It will of course still reload first a, then b, and then c. But for the relative local imports inside a, b, and c (if there are any) it does not know in which order it has to do them, so it just imports them in discovery order. For code where the import order of relative modules matters (which is not great code but also rather common), you have to write a manual reloader which reloads things in order (or just flush all relevant sys.modules entries).

External Integrations: For those connecting Cinema 4D with external dashboards, cloud assets, or custom web development systems, what protocols or lightweight local servers (e.g., WebSocket, REST endpoints) are you finding most reliable?

This is an extremely broad question which cannot be answered as is. You can make anything work at the end of day, it depends on what you want to do. When you want to connect web services with an UI (i.e., you have to represent the data somehow), I would recommend having a look at py_cmd_gui_html which showcases how to establish a two-way binding to an HTML page. The example runs the html page locally, but this could of course also be some service on the web. When you need an UI-less data transfer, I would probably go with REST and ship a bit better requests library with your plugin than what Python offers out of the box. You can technically also use the networking classes in the Maxon API Core, particularly UrlInterface but this is a low level API, not a convenience interfaces such as requests or urllib3 for Python, and more a barebones system like the bultin urllib of Python.

Cheers,
Ferdinand

ferdinand

Hello @mogh,

please excuse the delay. Without your plugin this is not easy to answer. Your GetState looks okayish to me, the GetActiveObjects and c4d_progressbar (I assume this is a helper for our status bar methods) calls are not great as GetState can be called quite often, but I would not expect this to be a huge bottleneck unless the scene is really large (GetActiveObjects implies traversing the scene).

The Message function however is not great. You free there without any safe guard (except for the document mode condition) your tool. Not only would the tool freeing then happen under a random message context, freeing a tool is also something that you can only do on the main thread (see the all new Scene Execution manual), as something could still work under the assumption that this tool is active. So, you would at least also have to add a main thread check there.

But even then I would consider this not great code, as this is a bit awkward place to free a tool. Both methods do not look essential to the operation of a plugin. Did you try to remove code or methods to see what exactly makes it slow?

Your second code listing looks much less rogue. As a tip I would still add an GeIsMainThreadAndNoDrawThread to your MSG_DOCUMENT_MODE_CHANGED condition. It is very likely that this message is only sent on the MT, but there can always be rogue plugins (or bad Maxon code) which violate such assumptions.

Cheers,
Ferdinand

I	i_mazlov
J	jana
M	maxon
F	ferdinand
M	m_adam