Debug Python related CRASH

[indexofrefraction]

hi, I rely heavily on Python Tags for some tasks

the setup is a bit complex, because the tags import my own modules,
but it worked reliable for years.

Now I get freezes / crashes on Windows when I open my file
and strangely that does not happen on macOS.

Is there a way to

1. have cinema4d log all python console output to a file?
   maybe I then can see what causes the crashes

2. safe start c4d without the execution of python tags in the scene?

best, index

[ferdinand]

Hi,

your problem sounds a bit like a violation of Cinema's Threading restrictions, but it is hard to tell without any code.

1. Not natively, but you can mess around with sys.stdout like in any Python interpreter. If you want to have both a console output and a log, you could use something like I did provide at the end of my post. Please remember that my code below is an example and not meant to be used directly.
2. Also not possible, afaik.

Cheers,
zipit

Code example for a logger object. You have to save the code to a file for this script to work. Details in the doc string.

"""A very simple logger that writes both to the console and to a file.

Saves this code to a file and run the file from the script manager. Cinema
will start logging all console output to a file in the same folder as
this script is located in.

The file will only be finalized / updated when you close Cinema, or when
you restart the logger, i.e. type the following into the console:

    import sys
    sys.stdout.restart()

To 'undo' what this script has done, either restart Cinema or type this
into the console:

    import sys
    sys.stdout = sys.stdout._stdout
"""

import datetime
import os
import sys

class ConsoleLogger(object):
    """A very simple logger that writes both to the console and to a file.

    The logger uses a dangling file (a file that is being kept open) which 
    is mostly considered a big no-no the days. The safer approach would be
    to use a file context in `write`, but reopening a file for each write
    access comes at a cost.

    The purpose of the whole class is to imitate file object so that it can
    be attached to sys.stdout. The relevant methods are ConsoleLogger.write
    and ConsoleLogger.flush. The rest is more or less fluff. 

    If you want to be on the safe side, you could also implement the whole
    file object interface, but I wasn't in the mood for doing that ;) 
    """
    def __init__(self):
        """Initializes the logger. 
        """
        self._file = None
        self._stdout = sys.stdout
        self.restart()

    def __del__(self):
        """Destructor.

        We have to close the file when the logger is getting destroyed. Also
        reverts sys.stdout to its original state upon instantiation.
        """
        sys.stdout = self._stdout
        self._file.close()

    def restart(self):
        """Starts a new logger session.
        """
        # Some date-time and file path gymnastics.
        time_stamp = datetime.datetime.now().ctime().replace(":", "-")
        file_path = "{}.log".format(time_stamp)
        file_path = os.path.join(os.path.split(__file__)[0], file_path)

        # Close old file and create a new one.
        if self._file is not None:
            self._file.close()
        self._file = open(file_path, "w+")

    def close(self):
        """Closes the logger manually.
        """
        self._file.close()
        self._file = None

    def write(self, message):
        """Writes a message.
        
        Writes both to the console `sys.stdout` object and the log file.
        
        Args:
            message (str): The message.
        
        Raises:
            IOError: When the log file has been closed.
        """
        self._stdout.write(message)
        if self._file is None:
            msg = "Logger file has been closed"
            raise IOError(msg)
        self._file.write(message)

    def flush(self):
        """Flushes sys.stdout.

        Needed for Python, you could also flush the file, but that is
        probably not what you want.
        """
        self._stdout.flush()

if __name__ == "__main__":
    sys.stdout = ConsoleLogger()

[indexofrefraction]

thx zipit,

can i apply that stdout redirection at the root somewhere, so that it applies for all python output?

somewhere in resource/modules/python/ ...

as my file crashes c4d on open, i need a temp hack to get that output

index

[indexofrefraction]

nevermind... i figured it out and added the console logger in

resource/modules/python/libs/python27/_maxon_init

to get live output in the file I also added

self._file.flush()

to the write() method

[indexofrefraction]

... the ConsoleLogger works nicely, but it didnt help me finding the bug
in the end it was this function, executed in a lot (±100) of python tags :

def userdataFocus(doc, tag):
	''' set object manager focus on userdata '''
	doc.SetActiveTag(tag)
	c4d.gui.ActiveObjectManager_SetObject(c4d.ACTIVEOBJECTMODE_TAG, tag, c4d.ACTIVEOBJECTMANAGER_SETOBJECTS_OPEN, c4d.ID_USERDATA)
	doc.SetActiveTag(None)

macos can handle that, on windows it leads to a crash.

I added it, because (most of the time) after opening a file the Python Tags "Tag" tab is active showing the code.
Actually a single function call would be enough, but how can you achieve that when the tags dont know of each other.

I'd need to store an info globally which doesn't stay alive when saving / reopening the file.
is that possible? (store info outside the python tags scope, accessible from all python tags)

[ferdinand]

Hi,

@indexofrefraction said in Debug Python related CRASH:

I'd need to store an info globally which doesn't stay alive when saving / reopening the file.

1. To store stuff 'globally' the common approach is to inject your objects into the Python interpreter. There are many places you can choose from and they are all equally bad. It's a hack. I like to use Python's module dictionary, because it has the benefit of a nice syntax (you can import your stuff). Just make sure not to overwrite any modules.

import sys

if "my_stuff" not in sys.modules.keys():
    sys.modules["my_stuff"] = "bob is your uncle!"

# Somewhere else in the same interpreter instance.
import my_stuff
print my_stuff

2. The life time of objects depends on their document being loaded. So if you want to 'cache' nodes beyond the life time of their hosting document, you have to do that manually. Below you will find a rough sketch on how you can approach that. There are many other ways to approach that. The UUIDs for a document will work beyond save boundaries, so you can cache something with that approach, save the document, even relocate it, and then load it again and feed the document into the cache interface query and will spit out the clones of the cached nodes.

Cheers,
zipit

"""A simple interface to cache nodes.

This is an example, not finished code.

Run in the script manager, read the comments. The *meat* is in
CacheInterface.add() and CacheInterface.query(), start reading in main().
"""

import c4d

class CacheInterface (object):
    """A simple interface to cache nodes.
    """
    def __init__(self):
        """Init.
        """
        self._cache = {}

    def _get_cache(self, uuid):
        """Returns the cache for a cache document identifier.

        Args:
            uuid (c4d.storage.ByteSequence): The document identifier for the
             cache.
        
        Returns:
            dict["doc": c4d.documents.BaseDocument, 
                 "nodes": list[c4d.GeListNode]]: The cache for the given 
                 identifier.
        """
        if not uuid in self._cache.keys():
            doc = c4d.documents.BaseDocument()
            self._cache[uuid] = {"doc": doc, "nodes": []}
        return self._cache[uuid]

    def _get_tag_index(self, node, tag):
        """Returns the index of a tag.

        Args:
            node (c4d.BaseObject): The object the tag is attached to.
            tag (c4d.BaseTag): The tag.
        
        Returns:
            int or None: The index or None when the tag is not attached to
             the object.
        """
        current = node.GetFirstTag()
        index = 0
        while current:
            if current == tag:
                return index
            current = current.GetNext()
        return None

    def add(self, node):
        """Adds a node to the cache.

        Args:
            node (c4d-GeListNode): The node to cache. Currently only 
            BaseObjects and BaseTags have been implemented.
        
        Returns:
            c4d-storage.ByteSeqeunce: The identifier for the cache the node
            has been inserted in.
        
        Raises:
            NotImplementedError: When an unimplemented node type has been 
            passed.
            ValueError: When the passed node is not attached to a document.
        """
        doc = node.GetDocument()
        if doc is None:
            raise ValueError("Can only cache nodes attached to a document.")

        uuid = doc.FindUniqueID(c4d.MAXON_CREATOR_ID)
        cache = self._get_cache(uuid)

        # BaseObjects
        if isinstance(node, c4d.BaseObject):
            # Pretty easy, clone and inject
            clone = node.GetClone(0)
            cache["doc"].InsertObject(clone)
            cache["nodes"].append(clone)
        # BaseTags
        elif isinstance(node, c4d.BaseTag):
            # For tags things are a bit more complicated.
            # First we get the index of the tag. 
            index = self._get_tag_index(node.GetObject(), node)
            if index is None:
                msg = "Could not find tag on source object."
                raise AttributeError(msg)

            # Then we clone the host of object of the tag and the tag.
            clone_object = node.GetObject().GetClone(0)
            clone_tag = clone_object.GetTag(node.GetType(), index)
            # Insert the host object into our cache document.
            cache["doc"].InsertObject(clone_object)
            # And append the tag of the same type at the given index
            # to our cache list. A bit dicey, you might wanna add some
            # checks to ensure that we actually got the right tag.
            cache["nodes"].append(clone_tag)
        # Other node types go here ...
        else:
            msg = "The node type {} has not been implemented for caching."
            raise NotImplementedError(msg.format(type(node)))
        return uuid

    def query(self, identifier):
        """
        Args:
            identifier (c4d.storage.ByteSeqence | 
            c4d.documents.BaseDocument): The identifier for the cache to
            query or the document that has been the source for the cache.
        
        Returns:
            list[c4d.GeListNode]: The cached nodes.
        
        Raises:
            KeyError: When the provided identifier does not resolve.
            TypeError: When the identifier is neither a document nor an uuid.
        """
        if isinstance(identifier, c4d.documents.BaseDocument):
            identifier = identifier.FindUniqueID(c4d.MAXON_CREATOR_ID)

        if not isinstance(identifier, c4d.storage.ByteSeq):
            msg = "Illegal identifier type: {}"
            raise TypeError(msg.format(type(uuid)))

        uuid = identifier
        if uuid not in self._cache.keys():
            msg = "Provided uuid is not a key in the cache."
            raise KeyError(msg)

        return self._cache[uuid]["nodes"]

def main():
    """Entry point.
    """
    if op is None:
        raise ValueError("Please select an object to cache.")
    # The cache
    interface = CacheInterface()
    # Some stuff to cache
    obj, tag = op, op.GetFirstTag()
    # Inject the nodes into the cache.
    uuid = interface.add(obj)
    # We do not need to store the uuid here, since both nodes come from
    # the same document.
    interface.add(tag)

    print "Original nodes:", obj, tag
    print "Their cache:", interface.query(uuid)

if __name__ == "__main__":
    main()

[indexofrefraction]

hi zipit,

thank u a lot for those tips !

about the CacheInterface i dont know if i do that right,
but if i select a cube and execute in the script manager
R20 crashes right away .-)

[ferdinand]

Hi,

that is a bit weird. I had ran the example code above in R21 without any crashes. The code is also quite tame, there isn't anything in there which I would see as an obvious candidate for conflicts, as it does not mess with scene or something like that.

Could you elaborate under which circumstances the code above produces a crash?

Cheers,
zipit

[m_adam]

Hi @indexofrefraction glad you found the cause of the issue, however I'm wondering in which context

def userdataFocus(doc, tag):
	''' set object manager focus on userdata '''
	doc.SetActiveTag(tag)
	c4d.gui.ActiveObjectManager_SetObject(c4d.ACTIVEOBJECTMODE_TAG, tag, c4d.ACTIVEOBJECTMANAGER_SETOBJECTS_OPEN, c4d.ID_USERDATA)
	doc.SetActiveTag(None)

Is called.

Since it does crash when you open the file directly, I assume you have the previous code called during the Execute of a Tag or a Python Generator.
But both functions are threaded functions. Meaning Any GUI related call is not allowed, because as you experienced, doing GUI stuff within a thread can make Cinema 4D crash for more information please read threading information

This could also confirm why it works on Mac OS and not on Windows since they are 2 different OS where the drawing pipeline is slightly different so you may found some cases where something does work in one Os but not on the other and vise-versa.

Cheers,
Maxime.

[indexofrefraction]

@zipit

just FYI, the CacheInterface immediately crashes R20 but works on R21

@m_adam

Ok, thanks!
I was doing this in every Python Tag and I have 100-200 of those in the File.
I don't really need to do this, it was just a "beauty" thing... that did end up ugly