Difficulty with cloners and and lifetime of objects

mastergog

Hey guys, using R23, Windows 10, C++.

I've got another question about my ObjectData plugin and GetVirtualObjects(). The workflow is that a material needs to be created and inserted into the document. Each object in the scene needs to have a its own material and its own file loaded into memory.

In general the problem is that I can't exactly track the lifetime of an object in C4D scene. Functions like ObjectData::Init() and ObjectData::Free() get called all over the place s o I don't rely on them.

My solution is to insert the material in GetVirtualObjects(). It consists of a different ObjectData "dummy" object and SceneHook object. The 2 ObjectData objects have a Link in their description pointing to each other. The second "dummy" ObjectData is inserted in the SceneHook:

GetVirtualObjects(BaseObject* object) {

        BaseObject* dummy = getAtomBaseLink<BaseObject>(*object, DUMMY_OBJECT, NONE, false).GetValue();
    	BaseObject* dummyParent = nullptr;
    	if (dummy ) {
    		DebugAssert(link->GetType() == ID_DUMMY_OBJECT);
    		dummyParent = getAtomBaseLink<BaseObject>(*dummy, PARENT_OBJECT, document, NONE, false).GetValue();
    	}
    
        if (!dummy || dummyParent != object) {
    
                BaseSceneHook* hook = document->FindSceneHook(HOOK_ID);
    		if (hook) {
    			BranchInfo branchInfo[1];
    			const Int32 count = hook->GetBranchInfo(branchInfo, COUNT_OF(branchInfo), NONE);
    			if (branchInfo[0].head) {
    				GeListHead *dummiesHead = branchInfo[0].head;
    				dummy  = BaseObject::Alloc(ID_DUMMY_OBJECT);
    				if (dummy != nullptr) {
                                       dummy->InsertUnderLast(dummiesHead);
                                       // set the links of the 2 objects to point to each other
                                       // do other stuff for the material
                               }
                       }
               }      
        }
}

At some points I iterate the dummies in the hook and if their Links are nullptr I know the original ObjectData object has been removed from the scene and I can free the material. That's the "garbage collection" process.

Overall this routine is working ok. The material is created just once for each object in the scene. But when I put my object in a Cloner, GetVirtualObjects(BaseObject* object) would get called 9 different times with 9 different objects. I'd get 9 different dummies and 9 different materials which is what we want. But every time a setting is changed GetVirtualObjects(BaseObject* object) would get called with yet another set of objects and add more materials, and this would continue indefinitely.
In the end my "garbage collection" process would clean them all up but still I really want to limit the creation to just the amount I need, but I'm not seeing how.

Furthermore I'm also keeping a hashmap in a different dll with some data, which corresponds to each object in the C4D scene. So i need some unique identifier, and I was using GetGUID but again it is not suitable for the situation with the cloners.

Thanks for the attention and any help will be appreciated!

Warm regards,
Georgi.

ferdinand

Hi @mastergog,

thank you for reaching out to us. We talked about your question and are struggling a bit with understanding its context or purpose. First of all I would like to point out some fundamental things:

You are modifying the scene graph in your GVO, which is a threaded context. This is not allowed.
As you already figured out yourself, the lifetime of a node, e.g. a BaseObject, is not as simple in Cinema as it might look like from the outside. While its GUI representation, for example in the object manger, might remain static, the actual node might be reallocated multiple times over the lifetime of its GUI representation. Which makes your whole approach not really fitting, as you are counting on that in your design from our understanding.

It would be helpful to understand in which context you are, so that I could better understand what you are trying to doing on a higher level. My general advice would be to move towards a hashmap-solution if you want to synchronize two scene graphs; Cinema's and yours for render engine for example. There are multiple ways how one can mark nodes in Cinema, one is Add/FindUniqueID, we just had another thread on the topic today. Just as for the user there, MAXON_CREATOR_ID might be helpful to you, because it comes with certain guarantees.

Happy holidays,
Ferdinand

mastergog

Hey thanks for the reply.

Yes I am aware. My workflow is with ExecuteOnMainThread() to be safe, just didn't include it in the details.

The crux of my question is exactly as you guessed. How to get unique ID's for each object so I can build a hashmap. These ID's should be consistent between reallocations of the node during the lifetime of its GUI representation. And they should be unique even for objects that are cloned.
The context is that I have volume caches which I load into ram using a separate DLL and I want to synchronize those with the objects in the scene. To load and free them just once, when it is most appropriate.

This Add/FindUniqueID is something I had missed so far and I'm looking at the previous threads as you advised. In some of the threads you refer to FindUniqueID returning a hash, but in C++ it seems I have to create the ID myself and add it to the object with AddUniqueID(Int32 appid, const Char* const mem, Int bytes) which include some extra parameters.
How would I go about letting Cinema create the hash?

Regards,
Georgi.

ferdinand

Hi @mastergog,

you can add your own unique IDs, but for that you have to register your own namespace so to speak, a plugin id fed into appid. What I was proposing was using the already existing IDs in the namespace MAXON_CREATOR_ID. The advantage here is that Cinema will create them for you Here is a little script which might give you some context.

Cheers,
Ferdinand

"""Create a bunch of objects and run the script, it will always spit out the 
same uuid for the nodes. If you run the script twice in quick succession, it 
will probably also spit out the same uuid for the caches. But when you invoke
a rebuild of a cache (by changing the parameter of a parametric object for 
example), the uuid of the cache of that object will change.
"""

import c4d

def get_uuid(node):
    """
    """
    # A simplified view on caches, we just assume here the top level node
    # of GetCache to be "our cache".
    cache = node.GetCache()

    # Get the unique ids stored under the namespace MAXON_CREATOR_ID for
    # the object and its cache.
    uuid_0 = bytes(node.FindUniqueID(c4d.MAXON_CREATOR_ID))
    if cache is not None:
        uuid_1 = bytes(cache.FindUniqueID(c4d.MAXON_CREATOR_ID))
    
    print (node)
    # These ids are just CRC-16 codes, so there is nothing really to see
    # here, but lets convert them to hex so that we can read them more
    # easily.

    # This is MAXON_CREATOR_ID for the node itself, it is always there and it
    # should be always the same even across reallocation boundaries, including
    # reloading a document. 
    print ("node uuid:", uuid_0.hex())

    # This is the cache of the node, it will also contain an MAXON_CREATOR_ID,
    # but it will NOT always be the same, since caches can change in "shape 
    # and form", so there is no way to maintain this continuity.
    # I assume this is what you are trying to do, or at least a good part of
    # it. This cannot be done, because you are basically asking here the 
    # caches to be static.
    if cache is not None:
        print ("cache uuid:", uuid_1.hex())

    # The only way I could think of would be some sort of hierarchy dependent
    # hashing for the nodes of a cache, i.e. determine the identity of a node
    # by its hierarchical relation to the cache root. But this would not be
    # really a solution either, because it would also assume caches to be
    # hierarchically static, which they are not in many cases.
    

def main():
    """Will go over all top level object nodes and print their unique ids.
    """
    node = op
    while node:
        get_uuid(node)
        node = node.GetNext()

if __name__=='__main__':
    main()

mastergog

@zipit yes thank you for the script. What I need seems to be exactly the Node uuid.

But I'm struggling with the C++ equivalent. I have this signature: Bool FindUniqueID(Int32 appid, const Char*& mem, Int& bytes ) const and I use it like so :

        const Char* value = nullptr;
	Int size = sizeof(Int32);
	if (object->FindUniqueID(MAXON_CREATOR_ID, value, size)) {
		/// do stuff with value
	}

I was expecting *(Int32*)value to be my unique ID but it is always the same. for all objects in the scene.
This is not the case your Python script, each node has its own ID, so I guess I'm using this wrong ?

Regards,
Georgi.

ferdinand

Hi @mastergog,

first of all, size is an output, not an input. Secondly, MAXON_CREATOR_ID is based on GeMarker which takes 16 bytes of memory. 8 bytes are the mac address of the machine, 4 bytes are a timestamp and the last 4 bytes are kind of a secret sauce. MAXON_CREATOR_ID just casts the memory block of a GeMarker into a maxon::Char array. So you could do something like this when looking at the data associated with MAXON_CREATOR_ID in C++, assuming you are going here for a human readable form:

// For https://developers.maxon.net/forum/topic/13090
static maxon::Result<void> pc13090(BaseDocument* doc)
{
	iferr_scope;
	// Get the first object in the scene.
	const BaseObject* node = doc->GetFirstObject();
	if (node == nullptr) {
		ApplicationOutput("No objects found in the scene.");
		return maxon::OK;
	}

	// Get its unique id stored under MAXON_CREATOR_ID and bring it into a human readable form.
	const Char* value = nullptr;
	Int size = 0;
	if (node->FindUniqueID(MAXON_CREATOR_ID, value, size)) {
		maxon::BaseArray<maxon::Char> uuid_representation;
		for (int i = 0; i < size; i++)
		{
			uuid_representation.Append(value[i]) iferr_return;
		}
		ApplicationOutput("First object's MAXON_CREATOR_ID.UID: '@'", uuid_representation);
	}
	return maxon::OK;
}

Cheers,
Ferdinand

mastergog

Hey @zipit happy holidays and thank you for the reply, I understand how this id works now.

However this ID seems to not be persistent for cloned objects. If I put my object in a cloner in instance mode grid array 3,1, 3 dimensions GetVirtualObjects(BaseObject* object...) would get called 9 times with 9 different objects. There are 9 different UUIDs and this is ok.

But then every subsequent call to GetVirtualObjects(BaseObject* object...), after I change some setting or simply move the object, there will be a different set of 9 UUIDs, so I can't really rely on them anymore. So with the cloner this reallocation boundary changes the UUIDs.

Is there a way around it?

EDIT: If I go in edit mode and change the settings of each instance in the cloner separately then the UUIDs seem to be persistent, but that's just in edit mode.

ferdinand

Hi @mastergog,

you are right, assuming you mean with cloned objects generators with caches. If you have a look at my first example in Python, you will see that I did cover this subject there and why it is conceptually not possible to do this. I also roughly lined out a workaround, hashing the objects in a cache by their hierarchical relations, which is in the end not really a workaround due to said conceptual limitations.

Cinema relies heavily on procedural geometry, you can track the hooks/stand-ins that represent that procedural something, but not what it is inside, because it is by definition polymorphic (in the sense of that that a cache can change its "shape"). You can establish a persistent link between two scene graphs at a given snapshot by simply expanding all caches in Cinema, but you cannot establish a persistent linkage down into the caches for an arbitrary time span.

Cheers and happy holidays,
Ferdinand

mastergog

@zipit said in Difficulty with cloners and and lifetime of objects:

# This is the cache of the node, it will also contain an MAXON_CREATOR_ID,
# but it will NOT always be the same, since caches can change in "shape 
# and form", so there is no way to maintain this continuity.
# I assume this is what you are trying to do, or at least a good part of
# it. This cannot be done, because you are basically asking here the 
# caches to be static.
if cache is not None:
    print ("cache uuid:", uuid_1.hex())

you are right, assuming you mean with cloned objects generators with caches

Ooooh maybe I misunderstood your point slightly.
My 9 objects in the cloner would always be different because they are part of the cache of the Cloner object.
If they stand on their own it is a different story, and the UUIds are persistent.

Hope i'm getting it right now ?

ferdinand

Hi @mastergog,

@mastergog said in Difficulty with cloners and and lifetime of objects:

Ooooh maybe I misunderstood your point slightly.
My 9 objects in the cloner would always be different because they are part of the cache of the Cloner object.
If they stand on their own it is a different story, and the UUIds are persistent.

Hope i'm getting it right now ?

yes, that is what I meant, my choice of words was probably not the best with "shape or form". Imagine a cache that has this form

root
    A
    B
    C

there is no mechanism in place which would make sure that the cache has always this "form", on the next update it could be

root
    B
    A
    C

What if there is an object added? What happens to nodes where the parent is being removed and so on. The purpose of a cache is to be rebuild. Each time this happens the old cache is our of the window. And even if Cinema would try to, there is no closed logic to copy over ids from the old to the new cache, i.e. to make ids on cache objects persistent.

Cheers,
Ferdinand

mastergog

@zipit yes ok seems I got this now, after completely disregarding the fact that object is part of the Cloner's cache.

So for my original problem of creating just X amount of materials for X amount of clones, that won't be doable. I'll just have to create as many as new objects are created and then just clean them all up at some point.

For the clean up mechanism, I have this dummy object which links to my original object and its material. The original object also links to the dummy object. The dummy objects are inserted in the a scene hook and at some point I iterate them and check if they link to nullptr object to determine whether the original object is still part of the document and whether to release the associated materials.

Do you think there's easier way to achieve this?

ferdinand

Hi @mastergog,

For the clean up mechanism, I have this dummy object which links to my original object and its material. The original object also links to the dummy object. The dummy objects are inserted in the a scene hook and at some point I iterate them and check if they link to nullptr object to determine whether the original object is still part of the document and whether to release the associated materials.

Depends on what you would define as easier, but you could create a TagData plugin whose node is hidden and then 'mark' all objects in a scene with these hidden tags. When an object is being deleted, and with it your hidden tag, NodeData::Free() is being called, allowing you to react to the event. This would probably be a bit more efficient than your method, but the problem is that you have the dangling sword of node reallocation over your head. So when a node is reestablished you would also would have to make sure if there already is a material/node counterpart on your side of things.

It would be probably not that easy, would have to try that myself. If performance is not an issue for you, I would stick with your working solution.

Cheers,
Ferdinand

mastergog

All right, thank you for all the help.

I'll close this now.