Importing pythonapi from ctypes freezes C4D
-
Hi there,
I've just noticed something slightly odd.. While doing the following:from ctypes import pythonapi, c_void_p, py_object
either in a script or plugin, Cinema4D freezes instantly.
I came across this issue, because I wanted to implement a way to react to my SpecialEventAdd() messages. I stumbled upon this example:
Is this still the way to go in 2024 to receive the SpecialEventAdd() p1 or p2 messages? And why is this instantly freezing C4D on my MacBook Air M2?
Cheers,
Lasse -
Hi @lasselauch it depends of the version you use, if its less than R23 which is bound to Python 2.7, you should use the method
GetCoreMessageParamOld
but if you use a newer version (which I hope) you should use GetCoreMessageParam23. You can find this information in the Python 3 Migration - Manual.Cheers,
Maxime. -
Hi @m_adam,
yes, that is correct. However, I experience a freeze everytime I do:
from ctypes import pythonapi
which I need either way. -
Hey @lasselauch,
Maxime is now on vacation (for quite some time), you will have to wait for his reply.
We talked this morning about your thread, and my take was that this is all out of scope of support because at least I would neither support
ctypes
nor third party code (the example you linked to). But Maxime wanted to support this, so it would be up to him.SpecialEventAdd
relies in C++ on casting your data, a concept that naturally does not translate well to Python. You can make stuff work, as the code demonstrates you linked to, but it is unnecessarily complicated and there are also quite some pitfalls inctypes
, most importantly threading. In Python you can send two integers withSpecialEventAdd
(not very useful), or anything you can fit into anint
usingstruct
(also not that useful). You should explain what you are trying to do, then we maybe can tell you how to achieve that.With all that being said, running
from ctypes import pythonapi
does not freeze my MacBook with 2024.4.0. You should also clarify which version you are using, you have tagged this posting asS24
but you are talking about2024
. But for everythingctypes
related you will have to wait for Maxime.Cheers,
Ferdinand -
Hi Ferdinand, thanks for the reply and I totally understand the out of scope of support part!
Thanks!
Regarding the freeze, my system specs are:
15", Apple M2 2023
macOS 13.5.2 (22G91)
C4D 2024.4.0
Just trying to import ctypes freezes C4D instantly.
Regarding the SpecialEventAdd() case:
I essentially want to act after a threaded operation is finished. So the last line of my threaded function calls:
c4d.SpecialEventAdd(ids.PLUGIN_ID_EXPORT)
To distinguish between different types of export I thought I could use the p1 (x) e.g.:
c4d.SpecialEventAdd(ids.PLUGIN_ID_EXPORT, x)
However when using this inside my dialog, I'm not sure how to receive the different p1 or p2 integers inside the CoreMessage():
def CoreMessage(self, id, msg): if id == ids.PLUGIN_ID_EXPORT: logger.info("PLUGIN_ID_EXPORT Message") if msg == x: logger.info("Finished thread Message") do_some_logic_here() return True return False
So what I'm asking is: I think different
int
are sufficient enough for my case, but I'm not sure how to receive those correctly within the CoreMessage or Message of a commanddata plugin.Thanks again,
Lasse -
Hey @lasselauch,
we will have a look at the crash, I will move the thread here into bugs when we can confirm it.
About your problem at hand: I have written a code example which should get you going. In short, the message data is stored under
BFM_CORE_PAR1
forp1
and_PAR2
forp2
. For the rest, please read the example. But I personally would still advise against taking such route unless one really has to.Make sure to set
USE_CTYPES
toFALSE
in the script when you still encounter issues importing it.Cheers,
FerdinandResult
Code
"""Demonstrates sending messages using both SpecialEventAdd() and a custom message stream implementation. The general idea of the MessageStream type is to have centralized place to store data as key-value pairs. In most cases, we can just define and expose such data container in a module which is imported by all plugins which have to share the data. I went here the slightly more complex route of the case where plugins are not able to do this, and all come with their own implementation and the data then being share via a "standard" module like sys, os, or what I picked here, c4d. This also demonstrates the most simple use case of SpeicalEventAdd(), CoreMessage() and ctypes, where we send an integer as the message data. As expressed in the thread above, I am not a big fan of this approach, this is all very un-pythonic and error prone. We technically COULD do all what MessageStream does and more here with ctypes alone, but the more complex our data gets, the more we will have to do to unpack our data from the wrapping capsule, and when we want to send anything else than ints, we will also have to start packing up data when sending them. This is all just a huge pain in the *** unless you are someone who is very comfortable with the Python C API and the memory layout of your sent data anyway. Note: What I have not done here, is made this thread safe. If we have multiple threads sending messages to the message stream, we will have to make sure that we do not run into access conflicts. This could either be done via calling c4d.threading.GeThreadLock() before accessing the message stream and .GeThreadUnlock() after, but that will really lock up everything which is not so great. An alternative would be to use a Python's threading.Lock(), threading.RLock() or threading.Semaphore() to lock the access to a message stream. The problem is there that our threads, c4d.threading. C4DThread, are not fully compatible with Python's threading, so be careful when using them together. But I have used threading.Semaphore() in the past without encountering any issues. """ USE_CTYPES: bool = True # Toggle for using ctypes and with it SpecialEventAdd() or not. Disable this # when you experience crashes when importing ctypes. IS_DEBUG: bool = True # Toggle for debug behavior. PID_MY_CORE_MESSAGE: int = 1063958 # A registered plugin ID to uniquely identify our core message. import c4d import time import inspect if USE_CTYPES: import ctypes from mxutils import CheckType class MessageStream: """Realizes a message stream that can be observed by others. An observer is just a delegate, i.e., a function that is called when 'something' happens in the message stream. Or in other words, more or less what the classic API message functions do in Cinema 4D. The main difference between the observer/delegate pattern and old school messages is that one has to manually register with the observable, here via AddObserver(). Can also be used as a plain message list without any observers, i.e., in conjunction with SpecialEventAdd() and CoreMessage(). I used here ints as message IDs, but we could also use strings or any other hashable type. I have done this solely so that the type works better in conjunction with SpecialEventAdd() which is int focused. See also: https://en.wikipedia.org/wiki/Observer_pattern Example: # Define a function that will be called when something happens in the message stream, # the observable in Maxon terms, we are interested in. def OnMessageStreamEvent(event: str, mid: int, data: any) -> bool: if event == MessageStream.EVENT_ADD: print (f"An item with ID {mid} was added to the message stream.") return True # Consume the event. return False # Otherwise do not consume the event. # Get the message stream with the handle 'main' and add our observer to it. stream: MessageStream = MessageStream.GetStream("main") stream.AddObserver(OnMessageStreamEvent) # Somewhere else and some time later we can now send messages to the stream. MessageStream.GetStream("main").Put(1, "Hello world!") # Will print: # An item with ID 1 was added to the message stream. """ EVENT_ADD: str = "ADD" # And item was added to the message stream. EVENT_REMOVE: str = "REMOVE" # An item was removed from the message stream. EVENT_FLUSH: str = "FLUSH" # The message stream was flushed. EVENT_UPDATE: str = "UPDATE" # An item in the message stream was updated. ATTACHMENT_POINT: object = c4d # The object/module to attach the global message streams to. STREAM_ATTRIBUTE: str = "__MESSAGE_STREAM_" # The prefix for the global message stream attributes. # --- Object model ----------------------------------------------------------------------------- def __init__(self, handle: str = ""): """Initializes the message stream with an optional handle. """ self._handle: str = CheckType(handle, str) self._data: dict[int, any] = {} self._observers: list[callable] = [] def __str__(self) -> str: """Returns a string representation of the message stream. """ return (f"<MessageStream('{self._handle}') at {hex(id(self)).upper()} with " f"{len(self._observers)} observers>") def __repr__(self) -> str: """Returns a string representation of the message stream. """ return str(self) def __len__(self) -> int: """Returns the number of messages in the message stream. """ return len(self._data) def __contains__(self, mid: int) -> bool: """Checks if a message ID is in the message stream. """ return mid in self._data def __iter__(self) -> iter: """Returns an iterator over the message stream. """ return iter(self._data) def __notify__(self, event: str, mid: int, data: any) -> None: """Notifies all observers of the message stream about an event until it is being consumed or until all observers have been notified. """ for observer in self._observers: if observer(event, mid, data): break def __getitem__(self, mid: int) -> any: """Returns the data of a message in the message stream. """ if mid not in self._data: raise KeyError(f"The message ID {mid} is not in the message stream.") return self._data[mid] def __setitem__(self, mid: int, data: any) -> None: """Puts a message into the message stream. """ isUpdate: bool = mid in self._data self._data[mid] = data self.__notify__(self.EVENT_UPDATE if isUpdate else self.EVENT_ADD, mid, data) def __delitem__(self, mid: int) -> None: """Removes a message from the message stream. """ if mid not in self._data: raise KeyError(f"The message ID {mid} is not in the message stream.") data: any = self._data[mid] del self._data[mid] self.__notify__(self.EVENT_REMOVE, mid, data) def __flush__(self) -> None: """Flushes the message stream. """ self._data.clear() self.__notify__(self.EVENT_FLUSH, 0, None) # --- Properties -------------------------------------------------------------------------------- @property def Handle(self) -> str: """Returns the handle of the message stream. """ return self._handle @property def Observers(self) -> tuple[callable]: """Returns a shallow copy of the observer list of the message stream. """ return tuple(self._observers) @property def Messages(self) -> dict[int, any]: """Returns a shallow copy of the messages of the message stream. """ return dict(self._data) # --- Observer pattern ------------------------------------------------------------------------- def AddObserver(self, observer: callable) -> None: """Adds a delegate to the message stream that is called when the message stream is modified. The delegate must have the signature observer(event: str, mid: int, data: any) -> bool. Returning true will consume the event, returning false will propagate it to the next observer. The event can be one of the EVENT_* symbols. The mid is the message ID and the data is the message data. """ if not callable(observer): raise TypeError("The observer must be a callable.") if observer in self._observers: raise ValueError("The observer is already registered.") # Make sure that the observer can in general deal with the required signature. try: sig: inspect.Signature = inspect.signature(observer) sig.bind(self.EVENT_ADD, 0, 0) except Exception as e: raise ValueError("The observer must have the signature observer(event: str, mid: int, " "data: any) -> bool.") from e self._observers.append(observer) def RemoveObserver(self, observer: callable) -> None: """Removes a delegate from the message stream. """ if observer not in self._observers: raise ValueError(f"The observer {observer} is not registered in the message " f"stream {self}.") self._observers.remove(observer) # --- Message stream --------------------------------------------------------------------------- def Flush(self) -> None: """Flushes the message stream. """ self.__flush__() def Put(self, mid: int, data: any) -> None: """Puts a message into the message stream. """ self[mid] = data def Get(self, mid: int, default: any = None) -> any: """Gets a message from the message stream. """ return self[mid] if mid in self else default def Remove(self, mid: int) -> None: """Removes a message from the message stream. """ del self[mid] # --- Static methods --------------------------------------------------------------------------- @staticmethod def GetStream(handle: str = "main") -> "MessageStream": """Returns the globally accessible message stream which has been attached to #ATTACHMENT_POINT under the given #handle. Doing it in this way is only necessary when we want to have a globally accessible message stream with multiple users which cannot share a module, otherwise we can just put this class in a file `stream.py`, declare there module level instance of this class and import this module in all plugins/modules which need it. ```stream.py class MessageStream: ... MY_STREAM: MessageStream = MessageStream("my_stream") ``` ```plugin.py from stream import MY_STREAM class MyPlugin (...): def __init__(self): MY_STREAM.AddObserver(self.OnMessageStreamEvent) ... ``` ```other_plugin.py from stream import MY_STREAM class OtherPlugin (...): def __init__(self): MY_STREAM.AddObserver(self.OnMessageStreamEvent) ... ``` """ attribute: str = f"{MessageStream.STREAM_ATTRIBUTE}_{handle.upper()}__" if not hasattr(MessageStream.ATTACHMENT_POINT, attribute): setattr(MessageStream.ATTACHMENT_POINT, attribute, MessageStream(handle)) stream: MessageStream = getattr(MessageStream.ATTACHMENT_POINT, attribute, None) if ((stream.__class__.__qualname__ != MessageStream.__qualname__) or (stream.Handle != handle)): raise ValueError(f"Could not get the message stream for the handle '{handle}'.") return stream @staticmethod def RemoveAllStreams() -> None: """Removes all globally accessible streams which have been attached to #ATTACHMENT_POINT. Doing this can become necessary in development, as we otherwise cannot reload/update the type when it is still in memory. """ for attr in dir(MessageStream.ATTACHMENT_POINT): if attr.startswith(MessageStream.STREAM_ATTRIBUTE): delattr(MessageStream.ATTACHMENT_POINT, attr) if IS_DEBUG: MessageStream.RemoveAllStreams() # --- Implementation of a message dialog which makes use of all this ------------------------------- class MessageDialog(c4d.gui.GeDialog): """Realizes a dialog that sends messages using both SpecialEventAdd() and the custom message stream implementation. """ # The IDs of the dialog elements. ID_BUTTON: int = 1000 ID_MESSAGE: int = 1001 def __init__(self): """Initializes the dialog by attaching an observer to the default custom message stream. """ MessageStream.GetStream().AddObserver(self.OnMessageStreamEvent) def CreateLayout(self) -> bool: """Sets up a UI to send a string message via the custom message stream. """ self.SetTitle("Message Dialog") self.GroupBorderSpace(10, 10, 10, 10) self.GroupSpace(5, 5) self.AddEditText(self.ID_MESSAGE, c4d.BFH_SCALEFIT, initw=200) self.AddButton(self.ID_BUTTON, c4d.BFH_CENTER, name="Send Message") self.SetString(self.ID_MESSAGE, "Hello world!") return True def Command(self, cid: int, msg: c4d.BaseContainer) -> bool: """Sets of both a message via the custom message stream and a core message when the user presses the "Send Message" button. """ if cid == self.ID_BUTTON: # Define the message ID as the current time in seconds and the message as the text in # the edit field. mid: int = int(time.time()) msg: str = self.GetString(self.ID_MESSAGE) # Send a message via the our custom message stream implementation. stream: MessageStream = MessageStream.GetStream() stream.Put(mid, msg) print (f"Sent message with ID {mid} to {stream}.") # And also invoke a core message using SpecialEventAdd() and ctypes, we just send here # the message ID as p1 while we rely on storing the actual message in the message stream. # We could also send more complex data here, but this can all get quite complex quite # quickly. if USE_CTYPES: c4d.SpecialEventAdd(PID_MY_CORE_MESSAGE, p1=mid, p2=0) print (f"Sent core message with the ID {PID_MY_CORE_MESSAGE} and data {mid}.") return True def OnMessageStreamEvent(self, event: str, mid: int, data: any) -> bool: """Called when a message is sent to the custom message stream implementation. """ print (f"An '{event}' event occurred for message with ID '{mid}' and data '{data}'.") return False # We do not consume the event. def CoreMessage(self, cid: int, msg: c4d.BaseContainer) -> bool: """Called by Cinema 4D when a core message is sent. """ # We receive a core message which notifies us that something has put something into our # custom message stream, the data which is being sent to us via the core message is just the # identifier of the message in our custom message stream. The core message data contains the # p1 and p2 values of the SpecialEventAdd() call which we must unpack here (just p1 in our # case). if cid == PID_MY_CORE_MESSAGE and USE_CTYPES: capsule: any = msg.GetVoid(c4d.BFM_CORE_PAR1) ctypes.pythonapi.PyCapsule_GetPointer.restype = ctypes.c_int ctypes.pythonapi.PyCapsule_GetPointer.argtypes = [ctypes.py_object] p1: int = ctypes.pythonapi.PyCapsule_GetPointer(capsule, None) # Do something with the message. print (f"Received core message with packed p1 data: {p1}") data: str | None = MessageStream.GetStream().Get(p1, None) print (f"Resolved to actual payload '{data}' for code message p1 payload '{p1}'.") return True if __name__ == "__main__": dlg: MessageDialog = MessageDialog() dlg.Open(c4d.DLG_TYPE_MODAL_RESIZEABLE)
-
Hey @lasselauch,
We are not able to reproduce this crash on an Intel, M1, or M3 MacBook with
2024.4.0
. Please provide and submit a crash report when this is still a problem for you. I would also recommend reinstalling Cinema 4D to rule out that your installation was damaged.Cheers,
Ferdinand