Identity of C4D objects in Python

Cairyn

Hello again;

here's an observation that I find interesting. Python and C++ use different object storage concepts (e.g. a simple type like integer is actually a reference in Python but a value in C++), so whatever object is constructed in C++ cannot be used 1:1 in Python (no memory mirroring).

That means for the Python API, every object returned by a function cannot be identical to the C++ object that C4D itself uses. Python would not understand the memory allocations there. So, the API function must return a proxy object that provides access for Python to the actual C++ objects. (Right?)

Here's a test (sorry for using the GeListHead again, this topic has nothing to do with my previous one):

import c4d

def main():
    obj = op
    prevHead = None
    print "---------------"
    while obj != None:
        head = obj.GetListHead()
        print type(head), id(head), head
        if prevHead != None:
            print prevHead == head, prevHead is head
        else:
            print "None"
        prevHead = head
        obj = obj.GetNext()

if __name__=='__main__':
    main()

The script goes through some objects in the Object Manager which are all on the same tree level, starting with the selected one, and check the GeListHead for each. The result is compared with the previous GeListHead by equality comparison == and identity comparison is.
The output looks like this:

---------------
<type 'c4d.GeListHead'> 1809691147248 <c4d.GeListHead object at 0x000001A559FF7BF0>
None
<type 'c4d.GeListHead'> 1809691146800 <c4d.GeListHead object at 0x000001A559FF7A30>
True False
<type 'c4d.GeListHead'> 1809691146928 <c4d.GeListHead object at 0x000001A559FF7AB0>
True False
<type 'c4d.GeListHead'> 1809691147248 <c4d.GeListHead object at 0x000001A559FF7BF0>
True False
>>>

Theoretically, the GeListHead should be the same object - identical, at the same address in memory - for all objects, as the objects belong to the same tree structure. That is not the case: the type is correct, but Python's own id() function returns a different number, and the memory address printed by the plain object printout is also different. These numbers repeat after three iterations, presumably because the garbage collection has destroyed the Python proxy by then and is reusing its properties.

It is logical that the equality test returns True (as the object is identical on the C++ side) but the identity test returns False (as the proxy object is different)... at least, it is logical in this interpretation.

Unfortunately, the objects should be identical too. If we were working in C++, the plain pointers would be the same, referencing the same object. The Python API adds an abstraction level which destroys the identity relation. I guess it would be necessary for the API to check for existing proxys and reuse those. (In the sample code, this would prevent the garbage collection from destroying the proxy until the very end of the script, as there would always be a reference to it through prevHead.)

Is that interpretation of mine even correct? I'm kind of reverse engineering here...

So, this raises first the question how to check for identity of two C4D objects from different references. The BaseObject has GetGUID() available for comparisons; a GeListHead as used here is no BaseObject though, and doesn't provide a unique ID.

Second, are there any more Python development details we have to be careful with, because the Python/C++ interface may introduce unforeseen difficulties?

e.g. memory allocation: C++ has an explicit allocation, while Python uses a garbage collector. Python will destroy an object when there are no more references to it (which in turn may destroy other objects that become non-referenced). Will that properly happen to a C4D object too? Like, I Remove() a BaseObject from a tree and then the variable where I store it goes out of scope... I haven't found evidence to the contrary, but it must be hellishly complicated to count references if the code changes between C++ modifications and Python modifications to the "same" object when methods of a Python plugin are executed.

e.g. destructor calls: when is the C4D object actually destroyed? The garbage collector may destroy the Python proxy later (C++: immediately) when it happens to run. Is the destructor of the C4D object called when the proxy is destroyed, or earlier (when the reference counter drops to 0), or later? If there are dependent objects (like tags for a BaseObject), when are these destroyed (provided the BaseObject C++ destructor takes care of them)?

e.g. Generator functions: these keep their state and execution point between calls, so all variables are preserved; including any proxy objects handling C4D objects. These are obviously vulnerable to changes between calls (okay, that's a bad example as Python has the same issue).

I've trying to find details on the C4D/Python interfacing in the Python manual but can't locate any deeper information.

PluginStudent

One way to "identify" objects in a C4D scene is to use the position in the object tree. That is what the "unique IP" system is using (GetUniqueIP(), GetUniqueID()).

ferdinand

Hi,

yes, that is an undocumented oddity of the Cinema 4D SDK. To test nodes for identity you have to use the equality operator and an equality comparison is performed by comparing the data containers of the nodes.

One related and rather annoying side-effect of this memory location peek-a-boo is that almost all types in Cinema are not hashable, i.e. do not implement __hash__, even those where you would not really expect it, like for example c4d.Vector (although vectors are testable for identity via is).

They probably should document hat more thoroughly.

Cheers,
zipit

m_adam

Hi @Cairyn you are right, our Python Object is in most of the time a holder of a C++ object pointer, and copy the python Object actually copy the pointer, not the pointed object which is way more optimized.

I would like to say that trust something on the pointer even in C++ is not really safe as Baseobject or GeListHead object can change. So that's why it recommended sticking to GetGUID/GeMarker/UniqueIP.

And all of this stuff is accessible through python, for GeMarker you can retrieve a BitSeq with C4DAtom.FindUniqueID(c4d.MAXON_CREATOR_ID) see Layers with the same name?.

For more information about how to identify objects, I let you read about Why are GUIDs not globally unique?.

But I would say in Python you have the same tool than in C++ except that you can access raw data, but you shouldn't trust pointer as they can change a lot (e.g. GetActiveObject() before and after an undo will not return the same pointed BaseObject).

Cheers,
Maxime.

Cairyn

@PluginStudent said in Identity of C4D objects in Python:

One way to "identify" objects in a C4D scene is to use the position in the object tree. That is what the "unique IP" system is using (GetUniqueIP(), GetUniqueID()).

Thanks for the suggestion - GetUniqueIP() is only available in BaseObject though. As far as I understand it, it is meant for the use with generators, so it's probably not "universal" enough to be used in arbitrary situations.

Cairyn

@m_adam Thank you for the confirmation and the reading suggestions.

Just for context, this is a general conceptual question and not connected to a specific code issue. I noticed the behavior while doing some test scripts with id() and is. It is worth noting that these Python properties need to be used carefully in concert with C4D objects.

(I wouldn't exactly recommend using pointer comparisons in C++ either )

I will include an "advanced" chapter in my Python/C4D book to mention this.