C4D Threading in python doesn't work as expect .

[Dunhou]

Hello everyone :

Problem :

I am working with a plugin that I want to listen to render queues elements status, and send a e-mail to me when they are all finished, I think the I should loop the queue and check status in backgroud(aka threading) to not lock the main GUI. So I find C4D Therading Class. But something going wrong and run out my knowledge.

The main problem is I search the therading document and github examples , it does have a Wait(Flase) function , but when I test the code , it will lock the main GUI anyway, when I leave Wait() function param "None" , It works but ask for a value in console. It does confuse me a lot . Did I do something wrong?

Cheers

Here is basic test codes :

import c4d
import time


class MyThread(c4d.threading.C4DThread):
    def __init__(self):
        self.count = 0

    # Called by self.TestBreak in Main
    def TestDBreak(self):
        if self.count == 5:
            return True
        return False

    # Called when MyThread.Start() is called
    def Main(self):

        # Iterates over 10
        for i in range(10):
            self.count = i + 1

            # Checks if the thread is asked to quit and call TestDBreak for custom breaking condition
            if self.TestBreak():
                print("Leaving")
                break

            print("Current:", i)
            time.sleep(1)


def main() :  
    thread = MyThread()  
    thread.Start()
    thread.Wait()
    #time.sleep(1)
    thread.End()  
  
main()

[ferdinand]

Hello @dunhou,

Thank you for reaching out to us. As a general warning, although you seem to be aware of it, I will quote the c4d.threading documentation:

[...] Please note that C4DThread, just as its native Python counterpart threading.Thread, is a mechanism to decouple the execution of code, but not to speed it up. Its intended use is to make computationally complex tasks non-blocking for the main-thread of Cinema 4D and by that ensure the responsiveness of the user interface. Parallelism in the sense of executing multiple parts of a larger task in parallel is currently not offered by the Python SDK [...]

There are only rarely occasions where such threading makes sense. Sending an email could be such a task when you want to wait for the response of the server for example. The reason why your code is not working as you want it to, is twofold:

1. You call (at least attempt to) C4DThread.Wait. This method will wait for your thread to finish. Which then will of course block all other execution of code in the meantime. Line 35 is only reached once your thread has counted to five and finished.
   
   • You circumvent this by adding a runtime error to your script by not adhering to the signature of Wait(bool).
   • This will cause line 34 to 36 never being executed. You end up with an orphaned thread wich is never being closed.
2. You are here in a Script Manager script which itself is blocking and not intended to run threads. While the script is running, you cannot interact with any GUI. The fact that it seemingly works for your thread.Wait() code variant, is because the script ends, or in your case is halted by an error, while the thread is still running.

Okay, what to do?

The general tool to deal with this is C4DThread.IsRunning. You call it to periodically check if the thread has finished. In your script, you could for example write:

def main() :  
    thread = MyThread()  
    thread.Start()
    while thread.IsRunning():
        c4d.StatusSetText(f"{thread.count = }")
    thread.End()
    c4d.StatusClear()

To update the status bar while your thread is running with its count value.

But you cannot escape the general blocking nature of a Script Manager script. You could technically open an async dialog from a Script Manager script, but that is just as bad as letting your thread run forever.

To use threading in a non-hacky way, you will need some kind of plugin, usually a CommandData plugin with an async dialog. In that async dialog you can then check periodically with either its Timer ,Message, or CoreMessage method if your thread has finished. The Py-TextureBaker example demonstrates this principle in Python.

Cheers,
Ferdinand

[m_adam]

Note that you can use a timer without a GeDialog by registering a MessageData plugin as shown in MessageData.

Chers,
Maxime.

[Dunhou]

Thanks for that explain @ferdinand @m_adam !

A long time hard work, sorry to reply so late.

I test for my purpose and it works as expected, thnaks again for your easy to read explain and a accompany tech explain , that helps a lot.

I haven't notice that script manager can block main GUI , I tried to check internet and download some files with thread before in script mananger and lock for a while , I thought I did a totally wrong way , maybe it's time to try this again

And by the way , I read the github link above and the extended version for teh baker plugin , the second one seems easier to understand for me .

And a liitle stupid question for MessageData , I have a rough understand about this , in my opinion, the MessageData plugin is doing a continue spying , like every 100ms check once . Does it slow down Cinema 4D like a thread run forever ? What's the diffirence between them?
(Don't sure if this will beyond the topic or the support range )

[ferdinand]

Hey @dunhou,

No, that question is completely valid. Yes, a GeDialog or MessageData which subscribes to a timer event will cost system resources. And when a user would have installed 100's of plugins using them, it would bring Cinema 4D to a crawl. But for a singular plugin there are no performance concerns.

But you also do not have to subscribe to a timer indefinitely. So, you can enable the timer once you have sent a mail and disable it once you have received a response for example. Find below a simple example for a message hook which manages URL request and shuts itself off once all requests have been processed. Adding new requests to it at runtime would also turn it on again.

Cheers,
Ferdinand

Result:

...
Timer message '2023-02-23 12:34:33.155866'
Timer message '2023-02-23 12:34:33.403811'
Timer message '2023-02-23 12:34:33.650063'
Successfully connected to 'https://www.python.org/'.
Failed to connect to 'https://www.google.com/foo' with status error: 'HTTP Error 404: Not Found'.
Timer message '2023-02-23 12:34:33.900924'
Successfully connected to 'https://www.google.com/'.
Failed to connect to 'https://www.foobar.com/' with status error: '<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1123)>'.
Timer message '2023-02-23 12:34:34.149718'
Timer message '2023-02-23 12:34:34.410978'
Timer message '2023-02-23 12:34:34.656376'
Timer message '2023-02-23 12:34:34.903763'
Timer message '2023-02-23 12:34:35.152023'
Timer message '2023-02-23 12:34:35.399493'
Timer message '2023-02-23 12:34:35.926809' # Here the hook has shut itself off.

Code:

"""Provides an example for a timed message hook which shuts itself off once all tasks are done.

This is simply done by returning a variable for MessageData.GetTimer, allowing the hook to turn its
timer subscription on and off based on the "tasks" it has.
"""

import c4d
import typing
import urllib.request
import datetime
import time

from http.client import HTTPResponse

class HttpRequestThread (c4d.threading.C4DThread):
    """Wraps an http(s) request in a thread.
    """
    def __init__(self, url: str, timeout: int = 10) -> None:
        """Initializes the thread with an url and timeout value.
        """
        if not isinstance(url, str):
            raise TypeError(f"{url = }")
        if not isinstance(timeout, int):
            raise TypeError(f"{timeout = }")

        self._url : str = url
        self._timeout: int = timeout
        self._result: typing.Union[HTTPResponse, Exception] = Exception("Unexpected null result.")
        
    def Main(self) -> None:
        """Makes the http(s) request.
        """
        # Postpone things a bit because otherwise all requests will have been finished before 
        # Cinema 4D has fully started and we would not see any timer messages before the threads
        # already did finish.
        time.sleep(5)

        # Make the request and store its error or result.
        try:
            self._result = urllib.request.urlopen(self._url, timeout=self._timeout)
        except Exception as e:
            self._result = e

    def GetResult(self) -> str:
        """Returns a string representation of the http(s) response for the request made by the thread.
        """
        if self.IsRunning():
            raise RuntimeError("This thread is still running.")
        
        res, url = self._result, self._url
        if isinstance(res, HTTPResponse) and res.status == 200:
            return f"Successfully connected to '{url}'."
        elif isinstance(res, HTTPResponse):
            return f"Failed to connect to '{url}' with status code '{res.status}'."
        else:
            return f"Failed to connect to '{url}' with status error: '{res}'."

        
class HttpRequestHandlerData (c4d.plugins.MessageData):
    """Realizes a message hook that runs http(s) request and reacts to their outcome.
    """
    PLUGIN_ID: int = 1060594

    def __init__(self) -> None:
        """Initializes the hook.

        Normally feeding such hook would be more complex and will include sending messages to convey
        new tasks. Here we just feed it a fixed amount of tasks on instantiation. 
        """
        self._httpRequestThreads: list[HttpRequestThread] = [
            # Should not fail before the sun does collapse into itself.
            HttpRequestThread(url=r"https://www.google.com/", timeout=10),
            HttpRequestThread(url=r"https://www.python.org/", timeout=10),
            # Will fail.
            HttpRequestThread(url=r"https://www.foobar.com/", timeout=10),
            HttpRequestThread(url=r"https://www.google.com/foo", timeout=10),
        ]
        for thread in self._httpRequestThreads:
            thread.Start()

        super().__init__()

    @property
    def PendingRequestCount(self) -> int:
        """Returns the number of pending request objects.
        """
        return len(self._httpRequestThreads)

    def ProcessRequests(self) -> None:
        """Exemplifies a callback function which is called by the timer.
        
        Here we use it to remove url threads once they have finished. Which in turn will mean
        that once there are no url threads anymore, the timer will be stopped. And once there
        is a new url thread, it will begin again.
        """
        result: list[HttpRequestThread] = []
        for thread in self._httpRequestThreads:
            if thread.IsRunning():
                result.append(thread)
                continue

            # Do something with a finished thread, we are just printing the result to the console.
            thread.End()
            print (thread.GetResult())

        # Set the new collection of still ongoing threads.
        self._httpRequestThreads = result

    def GetTimer(self) -> int:
        """Enables a timer message for the plugin with a tick frequency of ~250ms when there
        are pending requests and disables it when there are none.

        Note that the tick frequency will be approximately 250ms and not exactly that value. For
        values below 100ms the error will increase steadily and when setting the tick frequency to
        for example 10ms one might end up with a tick history such as this:
            [27ms, 68ms, 11ms, 112ms, 36ms, ...]
        """
        return 250 if self.PendingRequestCount > 0 else 0

    def CoreMessage(self, mid: int, _: c4d.BaseContainer) -> bool:
        """Called by Cinema 4D to convey core events.
        """
        if mid == c4d.MSG_TIMER:
            self.ProcessRequests()
            print (f"Timer message '{datetime.datetime.now()}'")
            
        return True

def RegisterPlugins() -> bool:
    """
    """
    if not c4d.plugins.RegisterMessagePlugin(
        HttpRequestHandlerData.PLUGIN_ID, "HttpRequestHandlerData", 0, HttpRequestHandlerData()):
        print (f"Failed to register: {HttpRequestHandlerData}")
        return False

if __name__ == "__main__":
    RegisterPlugins()

[Dunhou]

Thanks for this again ! @ferdinand

That help me understand more about MessageData and how to use the Timer in a 牧场more clever way .

And I get my code worked and run well.

Happy weekend !

[Dunhou]

@ferdinand Hello , a liitle furthur problem with the thread.

When I try to End the spy theard , it will lock the main GUI , untill the running task is done , and don't know why does this happened.

If more code is needed .I will send you an Email becouse it's a bit long(800 lines)

    def Abort(self):
        """Stop the Listening"""
        # Checks if there is a Thread process currently
        if self.RQListenerThread and self.RQListenerThread.IsRunning():
            if gui.QuestionDialog(Language.IDS_AskClose):
                self.aborted = True
                self.RQListenerThread.End() # False returns immediately although the thread will still run until it is finished
                self.RQListenerThread = None
                c4d.StatusSetText(f"Stop Listen to Render Queue")

Cheers~

[ferdinand]

Hey @dunhou,

yes, please send us the code via mail, because with the given code and explanation it would probably pure guess work for me. When you send us the code, please make sure to add a problem description in the form of:

1. Open Object Manager.
2. Add cube object.
3. Set Size.X to 500.

Result: Cinema 4D freezes.

The more precise you are with such things, the easier it is for us to reproduce them.

Cheers,
Ferdinand

[Dunhou]

@ferdinand thanks for your generoso help.

I already send you an email with an zip file , and some notes in the front of the codes.

Have a good day!