Defer Modifying Menus to the Main Thread

alexandre.dj

@ferdinand Thanks again for your help.

I managed to build our menus using a simple timer in the end, so it is waiting for the last c4d.C4DPL_BUILDMENU message before triggering the build.

I am however using a separate thread for this, and I encounter the issue you are mentionning in your answer, that is my menus are built but not showing up, as I am calling c4d.gui.UpdateMenus() from that separate thread, whereas it needs to be called from the main thread.

Is there a c4d method to send code to execute int he main thread? I can't find anything like this in the documentation. Or can I simply trigger a menu update (using a simple flag set to true when the side thread is done) from the main thread? There must be an Update() function somewhere that is run every few seconds?

edit: forked from building-menus-with-c4dpl_buildmenu-in-s26 by @ferdinand due to being off topic.

ferdinand

Hello @alexandre-dj,

Thank you for reaching out to us. This question of yours did deviate too far from the subject of building-menus-with-c4dpl_buildmenu-in-s26. Due to that, your posting has been forked. Note that our Forum Guidelines state:

A topic must cover a singular subject; the initial posting must be a singular question.

Users can ask follow-up questions, asking for clarification or alternative approaches, but follow-up questions cannot change the subject.

When the subject of the topic is "How to create a cube object?", then a follow-up question cannot be "How to add a material to that cube?" as this would change the subject.

A valid follow-up question would be "Are there other ways to create a cube object, as the proposed solution has the drawback X for my use-case?" or "Could you please clarify the thing Y you did mention in your answer, as this is still unclear to me?".

There is some leverage-room for this rule, but it is rather small. Small changes in the subject are allowed, large changes are not.

Please try to follow these rules in the future. It is certainly no catastrophe when users violate these from time to time, as it can be tricky to decide when something is on-topic or not. But we must enforce this rule to a reasonable degree, so that the forum remains a searchable knowledge base.

About your Question

In general, questions should be accompanied by code, as they otherwise tend to be very ambiguous with us having to play through multiple scenarios.

I am however using a separate thread for this, and I encounter the issue you are mentioning in your answer, that is my menus are built but not showing up, as I am calling c4d.gui.UpdateMenus() from that separate thread, whereas it needs to be called from the main thread. Is there a c4d method to send code to execute int he main thread?

In our C++ API there is the aptly named ExecuteOnMainThread. But I assume you are still on Python, right? The question here is a bit problematic since a CPython VM does not support true parallelism, due to the notorious global interpreter lock (GIL). This is why none of the jobqueue.h based mechanisms such as ExecuteOnMainThread have been ported to Python.

But I assume what you mean is, that you are being called from the non-main thread in C++ and must therefore adhere to the threading restrictions that come with it, such as for example not messing around with menus. While ExecuteOnMainThread would certainly be helpful here, such threading related concepts do not make too much sense in the context of the GIL. In C++, ExecuteOnMainThread is also a double-edged sword, since you either tie the calling thread to the main thread, by waiting for the result, or make the call not wait for the result, but then cannot rely on the fact that the main thread operation has already been carried out after the call.

I can't find anything like this in the documentation. Or can I simply trigger a menu update (using a simple flag set to true when the side thread is done) from the main thread? There must be an Update() function somewhere that is run every few seconds?

This is indeed more or less the solution to your problem. Cinema 4D has no Update function but there is the core message EVMSG_CHANGE which is effectively the same, since this core message is sent every time a scene has changed. You can receive core messages with the CoreMessage methods of the interfaces c4d.gui.GeDialog, c4d.gui.GeUserArea, and c4d.plugins.MessageData.

But in the end using a core message is more form than necessity. It is only important to defer your main thread bound operation to the main thread and you can use for that any method that you know will be called from the main thread at some point. You can then use c4d.threading.GeIsMainThreadAndNoDrawThread to determine if you are within a thread which is safe for any of the main thread bound operations. For modifying menus, checking with the slightly broader c4d.threading.GeIsMainThread should be enough.

Generally speaking, there is no guarantee that any method will always run on the main thread. Which is why there are these testing methods. All main thread bound tasks such as messages are usually sent from the main thread by Cinema 4D. And in for example a GeDialog.CreateLayout, MessageData.CoreMessage, or ObjectData.Message call you are therefore usually on the main thread. This is however only a convention and a rogue plugin could call your MyBaseObject.Message from anywhere and therefore cause a non-MT MyObjectData.Message call. In an ObjectData.GetVirtualObjects call on the other hand you will never be on the main thread because the scene evaluation is executed in parallel by the C++ layer.

Without knowing what you do, which plugin you implement, I cannot give you more concrete advice. You can also have a look at this topic as we talked there about deferring things to the main thread at the example of an ObjectData plugin modifying a scene.

Last but not least I am not sure that you need all this in the first place, since your posting could also be interpreted as such as you implemented your own C4DThread. You should share your code.

Cheers,
Ferdinand

alexandre.dj

Thank you Ferdinand and sorry about breaking the forum's rules. I will be more careful in the future and open a new ticket if needed.

alexandre.dj

@alexandre-dj
Here is my code. I created a BuildBufferedMenus class inheriting from threading.Thread and starting a counter running for 3 seconds.

Everytime a C4DPL_BUILDMENU message is emitted, I reset the counter to 0.

When the counter reaches 3 seconds, it builds the menus.

However, as discussed, the c4d.gui.UpdateMenus() call in my exit() function updates the menus but does not update the view. I cannot see them until I change the layout (minimize the window, change layout, or the like).

In my latest version, I added the menus_updated property flag to let the main thread know that my menus are built and that I need the layout to update. Hence my previous question about sending a message to the main thread or checking in any method occuring regularly that my menus need an update.

class BuildBufferedMenus(threading.Thread):
    """ Build c4d menus introducing a delay to deal with multiple builds (R26) """

    def __init__(self, max_time=3):
        super().__init__()
        self.max_time = max_time
        self.current_time = 0
        self.is_running = False
        self._menus_updated = False

    def build_menus(self):
        """ Method called from the plugin message to start or reset the timer """
        # start timer if thread not running
        if not self.is_running:
            self.start()
        # reset timer if thread is running
        else:
            self.reset()

    def run(self):
        """ Timer loop """
        self.is_running = True
        while self.current_time < self.max_time:
            self.current_time += 1
            time.sleep(1)
        # reinit timer variables and exit thread at the end
        self.is_running = False
        self.current_time = 0
        self.exit()

    def reset(self):
        """ Reset Timer """
        self.current_time = 0

    def exit(self):
        """ Exit timer """
        enhanceMainMenu()  # adds my custom menus
        c4d.gui.UpdateMenus()
        self._menus_updated = True

    @property
    def menus_updated(self):
        return self._menus_updated

    @menus_updated.setter
    def menus_updated(self, value):
        if not isinstance(value, bool):
            raise ValueError(f'{value} should be of type bool (True or False)')
        self._menus_updated = value

# create class instance
buffered_menus = BuildBufferedMenus()

def myPluginMessage(id, data):
    if id==c4d.C4DPL_BUILDMENU:
        buffered_menus.build_menus()

ferdinand

Hey @alexandre-dj,

No need to be sorry, but at some point, we must fork threads. Thank you for sharing your code. As already hinted at, I do not really see the necessity to use threading here. As shown in the other thread, I would simply check the menus if they already have been modified, by searching for markers of insertion such as, for example, a certain menu group being present. Alternatively, you could also just use a global bool didModifyMenu in your plugin. I assume the multiple C4DPL_BUILDMENU emissions are the reason for your threading attempts here.

But when you want to keep the threading for some reason, you will need a managing entity. Otherwise, threading does not make too much sense. That entity then usually checks on the state of a thread with C4DThread.IsRunning and then does something after that. The entity also often lives on, is being called from the main thread. We talked here about the subject and I also provided some example code.

In this case, I would (try to) use a MessageData hook. Starting this early in the app cycle with a thread which is meant to loop back into the main thread could be tricky. But here is how I would do it:

When C4DPL_BUILDMENU is being emitted, send a custom core message with c4d.SpecialEventAdd. The message ID should be a plugin ID of yours.
In MessageData.CoreMessage of a MessageData plugin of yours, listen for the plugin ID(s) send from (1.). When you receive the message, start your thread when it is not already running.
As shown in the threading example, turn on/off a timer event once the thread runs to parodically check its state.
Once the thread has finished, grab the menu it has built, and insert it into the main menu. When receiving a tick for a timer message, you should be on the main thread, but you should double check.

The problem with the whole approach is that modifying menus outside of the main thread is problematic because when you have this principal logic:

   app -- UpdateEvent --> [Thread(app.menu)] -- FinishEvent -->  app.menu = Thread.result

You risk writing outdated menu data, because in the time your thread has been cooking, something else could have updated app.menu and you write then an older state back. So, Thread should not modify the whole menu, as your code implies, but rather provide menu data which should be inserted.

But again, this whole threading idea seems unnecessarily complicated in this case.

Cheers,
Ferdinand

edit:

I initially overlooked that you are using the standard library threading type `threading.Thread`. You cannot do this when using the Cinema 4D Python API. You must use c4d.threading, specifically C4DThread, instead.

alexandre.dj

@ferdinand
Just a quick update that using an event message to force update the view after creating the menus is working great.

c4d.EventAdd(c4d.EVMSG_CHANGE)