Maxon Developers Maxon Developers
    • Documentation
      • Cinema 4D Python API
      • Cinema 4D C++ API
      • Cineware API
      • ZBrush GoZ API
      • Code Examples on Github
    • Forum
    • Downloads
    • Support
      • Support Procedures
      • Registered Developer Program
      • Plugin IDs
      • Contact Us
    • Categories
      • Overview
      • News & Information
      • Cinema 4D SDK Support
      • Cineware SDK Support
      • ZBrush 4D SDK Support
      • Bugs
      • General Talk
    • Unread
    • Recent
    • Tags
    • Users
    • Login

    Best plugin type for background (thread) processing?

    Cinema 4D SDK
    r20 python
    6
    16
    3.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      PluginStudent
      last edited by

      As the name says, the Wait() function waits for the thread to finish. Thus, it blocks the execution.

      What you need is a thread that exist independently from your UI (a CommandData plugin is a UI plugin). You can start that thread in a CommandData plugin, but this plugin must not own the thread.

      When the thread has finished, it can send a custom core message. You can catch that core message in a MessageData plugin. Then you can read the thread's data safely from the main thread.

      The C4DThread Manual gives some good overviews.

      1 Reply Last reply Reply Quote 0
      • P
        pim
        last edited by

        Thanks.
        I change the Wait() state to False, but that did not help.
        I guess, like you say, I should not do it in a command plugin.
        I will messagadata plugin a try.

        1 Reply Last reply Reply Quote 0
        • P
          pim
          last edited by

          I tried a messagedata plugin and I noticed that the messagedata is not running in a thread.
          When messagedata is running, cinema 4d is not reacting.
          I will now add the threading class UserThread(c4d.threading.C4DThread) as in above code.

          1 Reply Last reply Reply Quote 0
          • P
            pim
            last edited by

            Adding threading did not help.
            Cinema 4d is still not reacting during the time the messagedata runs.
            Even changing to Wait(False) did not help.

            class MYMESSAGEDATA(c4d.plugins.MessageData):   
                def CoreMessage(self, id, bc): 
                    if (id ==  PLUGIN_ID_MESSAGAPLUGIN):        
                        print "Message received."
            
                        thread = UserThread()
                        thread.Start()
                        thread.Wait(False)     # I tried True and False
                    return True	
            ``
            1 Reply Last reply Reply Quote 0
            • P
              PluginStudent
              last edited by

              Please read my post above and the C4DThread Manual carefully.

              You can start the thread in a CommandData plugin. But this plugin must not own the thread instance. The thread instance should be stored in the global scope.

              In the MessageData plugin, you should react to a core message sent from the thread. Nothing else; no creation or waiting.

              1 Reply Last reply Reply Quote 0
              • H
                heilei
                last edited by

                The examples Maxon has in GitHub include "Py-TextureBaker", which seems to use threads. I'm currently going through that code (and trying to understand it...), because I'm facing the exact same challenge as OP: how to start something in a thread, and make that thread send messages to other plugins.

                P 1 Reply Last reply Reply Quote 1
                • lasselauchL
                  lasselauch
                  last edited by

                  Yeah, I'm looking into this as well...

                  Ideally I want to run different function with a decorator: @execute_on_own_thread which would be very convenient.

                  1 Reply Last reply Reply Quote 1
                  • P
                    pim @heilei
                    last edited by

                    @heilei said in Best plugin type for background (thread) processing?:

                    Py-TextureBaker

                    That is a very good working example.
                    I rebuild my plugin using py-texturebaker and now it is working!

                    After some studying, I think @PluginStudent is fully correct when referring to the global scope.

                    @PluginStudent said in Best plugin type for background (thread) processing?:

                    Please read my post above and the C4DThread Manual carefully.

                    You can start the thread in a CommandData plugin. But this plugin must not own the thread instance. The thread instance should be stored in the global scope.

                    In the MessageData plugin, you should react to a core message sent from the thread. Nothing else; no creation or waiting.

                    I guess it is working because in the dialog the thread class is initiated.
                    textureBakerThread = None
                    and further in the code it is referenced with self.textureBakerThread.

                    I was using

                            thread = UserThread()
                            thread.Start()
                    

                    Everybody thanks for all the help.

                    1 Reply Last reply Reply Quote 1
                    • M
                      m_adam
                      last edited by

                      Hi @pim this is very different, texture baker will execute a specific command in a thread (So you execute it only at a given time) while in your case if I understood correctly you want something that runs in the background.

                      So here a general example that runs a thread in background and updates a GeDialog according to some random event. This is a very generic solution, in your case, the proper way would be to handle the thread creation, not within the C4DPL_PROGRAM_STARTED but in the init method of the GeDialog and end the thread in the AskClose method of the GeDialog. But since I wanted to demonstrate something very generic I used C4DPL_PROGRAM_STARTED and C4DPL_ENDACTIVITY so the thread will exist as long as Cinema 4D start and exit.

                      import c4d
                      import time, random, weakref
                      
                      PLUGIN_ID_COMMAND_DATA = 1000000
                      
                      # Store the global thread
                      GlobalThread = None
                      
                      # Store a weakref of the GeDialog
                      GlobalDialog = None
                      
                      # Background Thread is driven from Thread class to make it run in the background.
                      class BGThread(c4d.threading.C4DThread):
                          end = False
                      
                          # Called by TestBreak to adds a custom condition to leave
                          def TestDBreak(self):
                              return bool(self.end) 
                      
                          def GenerateBitmap(self):
                              "Generate a basebitmap with a random color"
                              clipMap = c4d.bitmaps.GeClipMap()
                              clipMap.Init(100, 100)
                              clipMap.BeginDraw()
                              clipMap.SetColor(int(random.random() * 255 ), int(random.random() * 255 ), int(random.random() * 255 ))
                              clipMap.FillRect(0, 0, 100, 100)
                              clipMap.EndDraw()
                      
                              # It's important to return a clone since the original basebtimap is owned by the GeClipMap.
                              # So if the GeClipMap is destructed (and it will be the case after the return statement), the BaseBtimap will also be destructed.
                              return clipMap.GetBitmap().GetClone()
                      
                          # Starts the thread to start listing to the port.
                          def Main(self):
                              while True:
                      
                                  # Checks if the thread needs to be closed e.g. a call of GlobalThread.End()
                                  if self.TestBreak():
                                      return
                      
                                  # Sleeps to not support 
                                  time.sleep(0.1)
                      
                                  # Retrieves the global variable and either check if there is no Dialog, or if the weakref is None
                                  global GlobalDialog
                                  if GlobalDialog is None or GlobalDialog() is None:
                                      continue
                      
                                  dlg = GlobalDialog()
                                  if not dlg.IsOpen():
                                      continue
                                  
                                  # Simulates a random update
                                  randomEvent = bool(random.randint(1, 100) <= 10)
                                  if randomEvent:
                                      bmp = self.GenerateBitmap()
                      
                                      # I use a try/Except block, because we are in a threaded environment, so even if I check previously if the dialog exists
                                      # it's not 100% thread-safe, since the user may have closed the UI between the previous stuff.
                                      try:
                                          GlobalDialog().UpdatePicture(bmp)
                                      except:
                                          pass
                      
                      # The Dialog class
                      class DialogWithBitmap(c4d.gui.GeDialog):
                          
                          def __init__(self):
                              global GlobalDialog
                              GlobalDialog = weakref.ref(self)
                      
                              self.bmp = None
                              self.customUiBmp = None
                      
                          # Define our Layout
                          def CreateLayout(self):
                              w = c4d.gui.SizePix(100)
                              h = c4d.gui.SizePix(100)
                      
                              bcBitmapButton = c4d.BaseContainer()
                              bcBitmapButton[c4d.BITMAPBUTTON_BUTTON] = True
                              bcBitmapButton[c4d.BITMAPBUTTON_BORDER] = c4d.BORDER_IN
                      
                              self.customUiBmp = self.AddCustomGui(1000, c4d.CUSTOMGUI_BITMAPBUTTON, "", c4d.BFH_FIT|c4d.BFV_TOP, w, h, bcBitmapButton)    
                              return True
                      
                          def UpdatePicture(self, bmp):
                              # Called from the thread
                              self.bmp = bmp
                      
                              # Sanity checks if the dialog is opened or eitehr the customUI is not defined
                              if self.IsOpen() and self.customUiBmp is not None:
                      
                                  self.customUiBmp.SetImage(self.bmp)
                                  if c4d.threading.GeIsMainThread():
                                      self.customUiBmp.Redraw()
                                  else:
                                      # If we are not on the main thread, redraw is not allowed, so add a special event with a unique ID
                                      c4d.SpecialEventAdd(PLUGIN_ID_COMMAND_DATA, 0, 0)
                              return True
                      
                          def CoreMessage(self, id, msg):
                              if id == PLUGIN_ID_COMMAND_DATA:
                                  self.customUiBmp.Redraw()
                              return True
                      
                      
                      # The CommandData plugin which host the Dialog object alive
                      class CommandDataDlg(c4d.plugins.CommandData):
                          dialog = None
                      
                          def Execute(self, doc):
                              if self.dialog is None:
                                  self.dialog = DialogWithBitmap()
                              return self.dialog.Open(dlgtype=c4d.DLG_TYPE_ASYNC, pluginid=PLUGIN_ID_COMMAND_DATA, xpos=-1, ypos=-1, defaultw=200, defaulth=150)
                      
                          def RestoreLayout(self, sec_ref):
                              if self.dialog is None:
                                  self.dialog = DialogWithBitmap()
                              return self.dialog.Restore(pluginid=PLUGIN_ID_COMMAND_DATA, secret=sec_ref)
                      
                      
                      # Handle our thread here
                      def PluginMessage(id, data):
                          global GlobalThread
                      
                          # At the start of Cinema 4D We lunch our thread
                          if id == c4d.C4DPL_PROGRAM_STARTED:
                              GlobalThread = BGThread()
                              GlobalThread.Start()
                      
                          # At the end we clean our Thread + BC used to pass data
                          elif id == c4d.C4DPL_ENDACTIVITY:
                              if GlobalThread:
                                  GlobalThread.end = True
                                  GlobalThread.End()
                      
                      
                      # Main function
                      def main():
                          c4d.plugins.RegisterCommandPlugin(PLUGIN_ID_COMMAND_DATA, "GeDialog Example", 0, None, "GeDialog Example", CommandDataDlg())
                      
                      # Execute main()
                      if __name__=='__main__':
                          main()
                      

                      However your suggestion @lasselauch is very nice, so I will take care of it when I will come back from vacation.
                      Note that due to the nature of Python, there is no real Multi-threading so don't expect performance boost, ( in reality you will only have a decreasing of performance, its better to use async stuff). The scope of thread in python is like the one in Example where you have something multiple things to do at the same time without the need to block everything, like GUI.
                      I'm going to leave on vacation and should come back Monday, I will try to clean up the previous code and make it as an example and also an example about your decorator idea @lasselauch.

                      Cheers,
                      Maxime.

                      MAXON SDK Specialist

                      Development Blog, MAXON Registered Developer

                      P 1 Reply Last reply Reply Quote 1
                      • lasselauchL
                        lasselauch
                        last edited by

                        Nice, thanks for the example @m_adam ! Have a great vacation and looking forward to the decorator example!

                        BTW – This example might come in handy if we want to integrate animated images, kinda a like a gif but based on an image-sequence... hmmm, I have to think about this! 🙂

                        Thanks & Cheers,
                        Lasse

                        1 Reply Last reply Reply Quote 0
                        • P
                          pim @m_adam
                          last edited by

                          @m_adam said in Best plugin type for background (thread) processing?:

                          ( in reality you will only have a decreasing of performance, its better to use async stuff).

                          Thanks for the example.
                          Could you also clarify a bit more for me, how to improve performance (async stuff)?

                          M 1 Reply Last reply Reply Quote 0
                          • M
                            mp5gosu
                            last edited by

                            @m_adam I have a question about the following portion of your code:

                                # At the end we clean our Thread + BC used to pass data
                                elif id == c4d.C4DPL_ENDACTIVITY:
                                    if GlobalThread:
                                        GlobalThread.end = True # 1
                                        GlobalThread.End() # 2
                            

                            Aren't 1 & 2 not redundant in this case?
                            TestDBreak() tests for self.end which is satisfied in the above example and therefore TestBreak() also returns True, so the thread exists.
                            Why the additional call to End()?

                            What am I missing here?

                            Cheers,
                            Robert

                            1 Reply Last reply Reply Quote 0
                            • M
                              m_adam @pim
                              last edited by m_adam

                              Hi @mp5gosu
                              Correct, it was just to demonstrate a possible usage.
                              The next code would not have the same sense since .end will be checked only the next TestDBreak is called, while End(False) will force the thread to be ended right now.

                              GlobalThread.end = True # 1
                              GlobalThread.End(False) # 2
                              

                              @lasselauch Here an implementation example of an execute_on_own_thread decorator.

                              import c4d
                              import functools
                              import time
                              import random
                              
                              global globalThreadStorage
                              globalThreadStorage = []
                              
                              
                              class BGThread(c4d.threading.C4DThread):
                              
                                  def __init__(self, func, *args, **kwargs):
                                      self.func = func
                                      self.args = args
                                      self.kwargs = kwargs
                              
                                  def Main(self):
                                      self.func(*self.args, **self.kwargs)
                                      return
                              
                              
                              class CleanupThread(c4d.threading.C4DThread):
                              
                                  def Main(self):
                                      while True:
                                          time.sleep(3)
                              
                                          c4d.threading.GeThreadLock()
                                          global globalThreadStorage
                              
                                          # If there is only 1 thread in the global list and its self, leave since there is nothing else to clean.
                                          if len(globalThreadStorage) == 1 and globalThreadStorage[0] == self:
                                              c4d.threading.GeThreadUnlock()
                                              return
                              
                                          # Removes all threads that are finished, so the garbage collector can delete them.
                                          for thread in reversed(globalThreadStorage):
                                              if not thread.IsRunning():
                                                  thread.End(False)
                                                  globalThreadStorage.remove(thread)
                              
                                          c4d.threading.GeThreadUnlock()
                              
                                          if self.TestBreak():
                                              return
                              
                                      return
                              
                              
                              # Creates the decorator with argument
                              def execute_on_own_thread(wait=False):
                              
                                  # The decorated function
                                  def decorated(func):
                                      @functools.wraps(func)
                              
                                      # The wrapper
                                      def wrapper(*args, **kwargs):
                                          thread = BGThread(func, *args, **kwargs)
                                          thread.Start()
                              
                                          # If we don't wait for the thread to be finalized, that means the created thread should live even after
                                          # this function is executed. So we store a thread reference in a global list and we also initialize
                                          # the cleanup thread, which as its name suggest will have the role to delete other thread.
                                          if not wait:
                                              c4d.threading.GeThreadLock()
                              
                                              # Adds the thread that encapsulates the function to the global list
                                              global globalThreadStorage
                                              globalThreadStorage.append(thread)
                              
                                              # Checks if there is already a cleanup thread, and if not create it
                                              cleanupThreadPresent = False
                                              for thread in globalThreadStorage:
                                                  if isinstance(thread, CleanupThread):
                                                      cleanupThreadPresent = True
                                                      break
                              
                                              if not cleanupThreadPresent:
                                                  cleanupThread = CleanupThread()
                                                  cleanupThread.Start()
                                                  globalThreadStorage.append(cleanupThread)
                              
                                              c4d.threading.GeThreadUnlock()
                              
                                          if wait:
                                              thread.End(wait)
                                              del thread
                              
                                          return
                                      return wrapper
                                  return decorated
                              
                              
                              @execute_on_own_thread(False)
                              def randomAction(a):
                                  rTime = random.random() * 5
                                  time.sleep(rTime)
                                  print(a, rTime)
                              
                              
                              def main():
                                  for x in xrange(10):
                                      randomAction(x)
                              
                                  print("main done")
                              
                              # Execute main()
                              if __name__=='__main__':
                                  main()
                              

                              @pim Here a safer and appropriate version for a GeDialog background thread, this does not rely on a global variable but only live as long as the dialog object lives while my previous example was something genric.

                              import c4d
                              import time, random, weakref
                              
                              PLUGIN_ID_COMMAND_DATA = 1000000
                              
                              # Background Thread is driven from Thread class to make it run in the background.
                              class BGThread(c4d.threading.C4DThread):
                              
                                  def __init__(self, dlg):
                                      self.dlg = weakref.ref(dlg)
                              
                                  def GenerateBitmap(self):
                                      # Generate a basebitmap with a random color"
                                      clipMap = c4d.bitmaps.GeClipMap()
                                      clipMap.Init(100, 100)
                                      clipMap.BeginDraw()
                                      clipMap.SetColor(int(random.random() * 255), int(random.random() * 255), int(random.random() * 255))
                                      clipMap.FillRect(0, 0, 100, 100)
                                      clipMap.EndDraw()
                              
                                      # It's important to return a clone since the original basebtimap is owned by the GeClipMap.
                                      # So if the GeClipMap is destructed (and it will be the case after the return statement), the BaseBtimap will also be destructed.
                                      return clipMap.GetBitmap().GetClone()
                              
                                  # Starts the thread to start listing to the port.
                                  def Main(self):
                              
                                      while True:
                                          # Creates a global try/except block since at any time the dlg can be destructed.
                                          try:
                                              # Checks if the thread needs to be closed e.g. a call of GlobalThread.End()
                                              if self.TestBreak():
                                                  return
                              
                                              # Sleeps to not bother too much the CPU
                                              time.sleep(0.1)
                              
                                              if not self.dlg().IsOpen():
                                                  continue
                              
                                              # Simulates a random update
                                              randomEvent = bool(random.randint(1, 100) <= 10)
                                              if randomEvent:
                                                  bmp = self.GenerateBitmap()
                                                  self.dlg().UpdatePicture(bmp)
                              
                                          # Catch the case self.dlg() return None, because the dlg object is destructed.
                                          # This should not happen because in the dialog.__del__ method, this thread is ended.
                                          # This also why a weakref is used, cause if the thread store directly a ref to the GeDialog,
                                          # Due to Python refcounting, the Dialog will never be deleted, so __end__ will never be called.
                                          except AttributeError:
                                              return
                              
                              
                              # The Dialog class
                              class DialogWithBitmap(c4d.gui.GeDialog):
                              
                                  def __init__(self):
                                      self.thread = BGThread(self)
                                      self.thread.Start()
                              
                                      self.bmp = None
                                      self.customUiBmp = None
                              
                                  def __del__(self):
                                      self.thread.End()
                              
                                  # Define our Layout
                                  def CreateLayout(self):
                                      w = c4d.gui.SizePix(100)
                                      h = c4d.gui.SizePix(100)
                              
                                      bcBitmapButton = c4d.BaseContainer()
                                      bcBitmapButton[c4d.BITMAPBUTTON_BUTTON] = True
                                      bcBitmapButton[c4d.BITMAPBUTTON_BORDER] = c4d.BORDER_IN
                              
                                      self.customUiBmp = self.AddCustomGui(1000, c4d.CUSTOMGUI_BITMAPBUTTON, "", c4d.BFH_FIT | c4d.BFV_TOP, w, h,
                                                                           bcBitmapButton)
                                      return True
                              
                                  def UpdatePicture(self, bmp):
                                      # Called from the thread
                                      self.bmp = bmp
                              
                                      # Sanity checks if the dialog is opened or eitehr the customUI is not defined
                                      if self.IsOpen() and self.customUiBmp is not None:
                              
                                          self.customUiBmp.SetImage(self.bmp)
                                          if c4d.threading.GeIsMainThread():
                                              self.customUiBmp.Redraw()
                                          else:
                                              # If we are not on the main thread, redraw is not allowed, so add a special event with a unique ID
                                              c4d.SpecialEventAdd(PLUGIN_ID_COMMAND_DATA, 0, 0)
                                      return True
                              
                                  def CoreMessage(self, id, msg):
                                      if id == PLUGIN_ID_COMMAND_DATA:
                                          self.customUiBmp.Redraw()
                                      return True
                              
                              
                              # The CommandData plugin which host the Dialog object alive
                              class CommandDataDlg(c4d.plugins.CommandData):
                                  dialog = None
                              
                                  def Execute(self, doc):
                                      if self.dialog is None:
                                          self.dialog = DialogWithBitmap()
                                      return self.dialog.Open(dlgtype=c4d.DLG_TYPE_ASYNC, pluginid=PLUGIN_ID_COMMAND_DATA, xpos=-1, ypos=-1,
                                                              defaultw=200, defaulth=150)
                              
                                  def RestoreLayout(self, sec_ref):
                                      if self.dialog is None:
                                          self.dialog = DialogWithBitmap()
                                      return self.dialog.Restore(pluginid=PLUGIN_ID_COMMAND_DATA, secret=sec_ref)
                              
                              
                              # Main function
                              def main():
                                  c4d.plugins.RegisterCommandPlugin(PLUGIN_ID_COMMAND_DATA, "GeDialog Example", 0, None, "GeDialog Example",
                                                                    CommandDataDlg())
                              
                              
                              # Execute main()
                              if __name__ == '__main__':
                                  main()
                              

                              Regarding async, I can only forward you to Async IO in Python: A Complete Walkthrough. The main idea is Python does not support MultiThreading, this is the nature of the GIL so only one instruction can be executed in the Python VM machine at a given time. So as an example I write to a file a very very long string of few Gb, The python call is pretty short, what takes the most time is the write into the file (hardware limitation) so instead to wait until the file is written with async stuff you tell, to the python VM it can execute some stuff during this lost time of waiting for hardware (this is a very rough explanation). However, note this is not well supported in Python2.7 but this is in Python3.

                              Cheers,
                              Maxime.

                              MAXON SDK Specialist

                              Development Blog, MAXON Registered Developer

                              1 Reply Last reply Reply Quote 3
                              • lasselauchL
                                lasselauch
                                last edited by m_adam

                                Whoooooooooooop, whoooooooooooop!

                                Thank you so much, @m_adam !!!

                                1 Reply Last reply Reply Quote 0
                                • M
                                  m_adam
                                  last edited by

                                  Since this topic is older than a week I marked is as closed, but feel free to reopen it if you have further questions.

                                  Cheers,
                                  Maxime

                                  MAXON SDK Specialist

                                  Development Blog, MAXON Registered Developer

                                  1 Reply Last reply Reply Quote 0
                                  • B bentraje referenced this topic on
                                  • First post
                                    Last post