BaseTake AddTake

[JohnTerenece]

Hi.

I am currently working on an object plugin that would utilize Takes for a user to be able to quickly iterate through different potential versions of their scene. I've looked through the sdk and the different manuals that are available and I believe I understand the processes required.

My issue comes in with the amount of time it take Cinema to run my plugin in regards to the creation of the Takes and if this is just the time that it takes to create them.

I created a simple scene with a hundred objects underneath my plugin. With the code below I create a Take and assign a layer to each of the child objects. For time testing I create an additional 500 Takes using the first Take as the cloneFrom inside AddTake. This isn't a final version of my code just a test example.

The code is executed on a button press.

			case idRunTakes:
			{
				TakeData* takeData = doc->GetTakeData();
				if (!takeData)
					return TRUE;

				LayerObject *hideLayer = LayerObject::Alloc();

				const Vector hsv = Vector(1.0, 0, 0);
				const Vector rgb = HSVToRGB(hsv);
				newTakeTime = 0;
				settingTakeTime = 0;
				LayerData newdata;
				newdata.color = rgb;
				newdata.solo = FALSE;
				newdata.view = FALSE;
				newdata.render = FALSE;
				newdata.manager = TRUE;
				newdata.locked = FALSE;
				newdata.generators = FALSE;
				newdata.deformers = FALSE;
				newdata.expressions = FALSE;
				newdata.animation = FALSE;
				newdata.xref = TRUE;
				hideLayer->SetLayerData(doc, newdata);
				hideLayer->SetName("Hide Layer"_s);

				GeListHead* layerList = NULL;
				layerList = doc->GetLayerObjectRoot();

				layerList->InsertLast(hideLayer);
				maxon::BaseArray<BaseObject*> childObjArray;
				GatherAllChildObjects(splineObj->GetDown(), childObjArray);

				DescID aliasLinkDId = DescLevel(ID_LAYER_LINK, DA_ALIASLINK, 0);
				GeData setData;
				setData.SetBaseList2D((BaseList2D*)hideLayer);

				Float createNewTakesTime = 0;
				Float timeStart = 0;
				Float loopTimeStart = GeGetMilliSeconds();

				timeStart = GeGetMilliSeconds(); 
				BaseTake* newTake = takeData->AddTake(String("Take " + String::IntToString(0)), nullptr, nullptr);
				if (newTake == nullptr)
					return TRUE;

				createNewTakesTime = createNewTakesTime + GeGetMilliSeconds() - timeStart;
				for (Int32 childObjIndex = 0; childObjIndex < childObjArray.GetCount(); childObjIndex++)
				{
					BaseOverride* overrideNode = newTake->FindOrAddOverrideParam(takeData, childObjArray[childObjIndex], aliasLinkDId, setData);
					if (overrideNode == nullptr)
						return TRUE;

					childObjArray[childObjIndex]->SetLayerObject(hideLayer);
					overrideNode->UpdateSceneNode(takeData, aliasLinkDId);
				}


				for (Int32 takeIndex = 1; takeIndex < 501; takeIndex++)
				{
					timeStart = GeGetMilliSeconds();
// This is the line taking up a lot of time
					BaseTake* loopTake = takeData->AddTake(String("Take " + String::IntToString(takeIndex)), nullptr, newTake);
					if (loopTake == nullptr)
						return TRUE;
					createNewTakesTime = createNewTakesTime + GeGetMilliSeconds() - timeStart;
					
				}
				ApplicationOutput("Take time " + String::FloatToString(createNewTakesTime) + "    " + String::FloatToString(GeGetMilliSeconds() - loopTimeStart - createNewTakesTime));
				break;
			}


// Code for getting all of the child objects
void TakeCreatorPlugin::GatherAllChildObjects(BaseObject *childObject, maxon::BaseArray<BaseObject*> &objChildOfNullsArray)
{
	if (childObject == nullptr)
		return;

	while (childObject)
	{
		objChildOfNullsArray.Append(childObject);


		GatherAllChildObjects(childObject->GetDown(), objChildOfNullsArray);
		childObject = childObject->GetNext();
	}
}

Running the code in my test scene gives the following print out "Take time 1920.243 20.232". So most of the time my plugin is running is taken up by adding the takes into the document. Is this just the time that Cinema takes in this kind of circumstance to create the Takes or am I just missing a crucial step,

I've looked through the manuals in the sdk and tried to follow them.

I've also tried to create all of the Takes empty and add the Overrides to each one individually which increases the time that step takes which isn't desirable either.

Any help would be greatly appreciated.

JohnTerenece

[JohnTerenece]

As an additional question when it comes to deleting the takes it seems to take a considerable amount of time to run DeleteTake on the above mentioned five hundred takes. The objects in the scene only have one overridden parameter each but it took over two minutes to run the code below to delete all of the takes using the code below running on a button press.

Float takeDeleteTimeStart = GeGetMilliSeconds();
				TakeData* takeData = doc->GetTakeData();
				if (!takeData)
					return TRUE;

				BaseTake* currentTake = takeData->GetCurrentTake();
				if (currentTake == nullptr)
					return TRUE;
				BaseTake *removeTakes = takeData->GetMainTake();
				if (removeTakes)
				{
					BaseOverride *removeOverride;
					BaseTake *deleteTake = removeTakes->GetDown();
					maxon::BaseArray<BaseTake*> arrayOfTakes;
					while (deleteTake)
					{

						arrayOfTakes.Append(deleteTake);
						deleteTake = deleteTake->GetNext();
					}

					DescID replaceId = DescLevel(ID_BASEOBJECT_XRAY, DTYPE_BOOL, 0);
					
					for (Int32 deleteIndex = 0; deleteIndex < arrayOfTakes.GetCount(); deleteIndex++)
					{
						takeData->DeleteTake(arrayOfTakes[deleteIndex]);
					}
					arrayOfTakes.Flush();

				}
				ApplicationOutput("Total time to dumb delete " + String::FloatToString(GeGetMilliSeconds() - takeDeleteTimeStart));

Not asking for someone to debug the code, just wondering if it taking so long to both create the Takes and delete them given the scene would be expected or if I'm missing something fundamental when it comes to Takes.

Any help would be greatly appreciated.

John Terenece

[ferdinand]

Hello @johnterenece,

thank you for reaching out to us and please excuse the delay.

I ran your code, and it is correct (except for some small stuff, find a listing of the code I used at the end of this posting). The problem you encounter is a weakness of the Takes system, specifically the method AddTake . As you indicated yourself, the problematic part is that you clone a take when adding your 500 takes. I.e., this instruction will cause your code to take ~20 seconds for a given document with an object hierarchy of twelve objetcs.

// This is the line taking up a lot of time
BaseTake*   loopTake = takeData->AddTake(FormatString("Take @", takeIndex), nullptr, newTake);
if (loopTake == nullptr)
    return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not allocate take"_s);

And changing it to this (i.e., newTake is not being cloned anymore), will only take ~0.15 seconds.

// This is the line taking up a lot of time
BaseTake*   loopTake = takeData->AddTake(FormatString("Take @", takeIndex), nullptr, nullptr);
if (loopTake == nullptr)
    return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not allocate take"_s);

The underlying behavior is here that AddTake scales linearly with the amount of takes that are in the take graph, which is very unfavorable. Due to how the take system is implemented, this is also mostly independent from the number of nodes you create take overrides for, i.e., you always must pay a high upfront cost. Here are some quick metrics for the execution time of AddTake() over the 500 calls in three different scenarios.

12 objects with                    12 objects without                2 objects with
copying a take                     copying a take                    copying a take
                                                                                      
Total time: 20.388 sec             Total time: 0.150 sec             Total time: 15.152 sec
----------------------             ---------------------             ----------------------
0: 24.468 ms                       0: 0.154 ms                       0: 2.465 ms
1: 12.391 ms                       1: 0.134 ms                       1: 1.446 ms
2: 12.359 ms                       2: 0.163 ms                       2: 1.487 ms
3: 11.588 ms                       3: 0.183 ms                       3: 1.484 ms
4: 11.621 ms                       4: 0.136 ms                       4: 1.499 ms
5: 11.446 ms                       5: 0.135 ms                       5: 1.525 ms
6: 11.543 ms                       6: 0.172 ms                       6: 1.548 ms
7: 11.489 ms                       7: 0.137 ms                       7: 1.546 ms
8: 11.380 ms                       8: 0.136 ms                       8: 1.573 ms
9: 11.647 ms                       9: 0.204 ms                       9: 1.567 ms
...                                ...                               ...
...                                ...                               ...
...                                ...                               ...
490: 92.778 ms                     490: 0.430 ms                     490: 83.169 ms
491: 91.689 ms                     491: 0.457 ms                     491: 81.300 ms
492: 92.615 ms                     492: 0.437 ms                     492: 82.658 ms
493: 93.333 ms                     493: 0.449 ms                     493: 82.237 ms
494: 93.359 ms                     494: 0.466 ms                     494: 82.746 ms
495: 93.755 ms                     495: 0.431 ms                     495: 84.097 ms
496: 93.189 ms                     496: 0.482 ms                     496: 84.764 ms
497: 93.985 ms                     497: 0.440 ms                     497: 84.372 ms
498: 93.967 ms                     498: 0.453 ms                     498: 93.413 ms
499: 94.172 ms                     499: 0.438 ms                     499: 83.808 ms

I have talked with the dev, and there were some suggestions for how to fix this from our side (which might have hinted at what you can do), but unfortunately the problem lies deeper from my findings. As an example, try deleting all 500 of these "heavy" takes, i.e., takes which contain overrides in one operation in Cinema 4D from the GUI. You will see that this takes quite some time.

My initial reaction when I saw your posting was that this problem is somehow to be expected, as creating, and updating such complex take graph will take time. I will push the subject back to the developer of the Take System and is then up to him to decide if we will consider this a bug.

I will update this thread with an outcome on this, but there is not much more what you can do.

Cheers,
Ferdinand

The code I did use:

// Testing some take related user code.
// 
// As discussed in:
//     https://developers.maxon.net/forum/topic/13864

#include "c4d_basedocument.h"
#include "c4d_general.h"
#include "lib_description.h"
#include "lib_takesystem.h"
#include "obaselist.h"

#include "maxon/apibase.h"
#include "maxon/lib_math.h"

// Code for getting all of the child objects
//
// [FH] This function cannot work as provided by you, as it was lacking the error handling for 
// BaseArrayInterface::Append(). I have just added a scope handler which silently exits the function
// on an error. This function should be of type maxon::Result<void> to properly propagate errors.
void GatherAllChildObjects(BaseObject* childObject, maxon::BaseArray<BaseObject*>& objChildOfNullsArray)
{
	iferr_scope_handler
	{
		return;
	};

	if (childObject == nullptr)
		return;

	while (childObject)
	{
		objChildOfNullsArray.Append(childObject) iferr_return;


		GatherAllChildObjects(childObject->GetDown(), objChildOfNullsArray);
		childObject = childObject->GetNext();
	}
}

static maxon::Result<void> pc13864(BaseDocument* doc)
{
	iferr_scope;

	TakeData* takeData = doc->GetTakeData();
	if (!takeData)
		return maxon::OK;

	// [FH] I disabled the automatic undo generation for takes, but this had no measurable impact for me.
	// takeData->SetUndoState(false);

	LayerObject* hideLayer = LayerObject::Alloc();

	const Vector hsv = Vector(1.0, 0, 0);
	const Vector rgb = HSVToRGB(hsv);

	// [FH] I found your timing/profiling a bit confusing, so I disabled it.
	
	// newTakeTime = 0;
	// settingTakeTime = 0;

	LayerData newdata;
	newdata.color = rgb;
	newdata.solo = FALSE;
	newdata.view = FALSE;
	newdata.render = FALSE;
	newdata.manager = TRUE;
	newdata.locked = FALSE;
	newdata.generators = FALSE;
	newdata.deformers = FALSE;
	newdata.expressions = FALSE;
	newdata.animation = FALSE;
	newdata.xref = TRUE;
	hideLayer->SetLayerData(doc, newdata);
	hideLayer->SetName("Hide Layer"_s);

	GeListHead* layerList = NULL;
	layerList = doc->GetLayerObjectRoot();

	// [FH] splineObj was not exposed by your code. 
	BaseObject* splineObj = doc->GetFirstObject();
	if (splineObj == nullptr)
		return maxon::UnexpectedError(MAXON_SOURCE_LOCATION, "No objects in document"_s);

	layerList->InsertLast(hideLayer);
	maxon::BaseArray<BaseObject*> childObjArray;
	GatherAllChildObjects(splineObj->GetDown(), childObjArray);

	DescID aliasLinkDId = DescLevel(ID_LAYER_LINK, DA_ALIASLINK, 0);
	GeData setData;
	setData.SetBaseList2D((BaseList2D*)hideLayer);

	//Float createNewTakesTime = 0;
	//Float timeStart = 0;
	//Float loopTimeStart = GeGetMilliSeconds();
	//timeStart = GeGetMilliSeconds();

	BaseTake* newTake = takeData->AddTake(String("Take " + String::IntToString(0)), nullptr, nullptr);
	if (newTake == nullptr)
		return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not allocate take"_s);

	// createNewTakesTime = createNewTakesTime + GeGetMilliSeconds() - timeStart;
	for (Int32 childObjIndex = 0; childObjIndex < childObjArray.GetCount(); childObjIndex++)
	{
		BaseOverride* overrideNode = newTake->FindOrAddOverrideParam(
			takeData, childObjArray[childObjIndex], aliasLinkDId, setData);
		if (overrideNode == nullptr)
			return maxon::UnexpectedError(MAXON_SOURCE_LOCATION, "Could not add override"_s);

		childObjArray[childObjIndex]->SetLayerObject(hideLayer);
		overrideNode->UpdateSceneNode(takeData, aliasLinkDId);
	}

	// [FH] Container for the timings per call/iteration.
	maxon::BaseArray<maxon::Float64> timings;

	for (Int32 takeIndex = 1; takeIndex < 501; takeIndex++)
	{
		const maxon::Float64 t = GeGetMilliSeconds();

		// timeStart = GeGetMilliSeconds();
		
		// This is the line taking up a lot of time
		BaseTake*	loopTake = takeData->AddTake(FormatString("Take @", takeIndex), nullptr, newTake);
		if (loopTake == nullptr)
			return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not allocate take"_s);

		// [FH] (not copying a take): This will drop the execution time from 20 sec to 0.15 sec on my 
		// machine, but will not set the layer for each object in the new take either of course.
		
		// BaseTake* loopTake = takeData->AddTake(FormatString("Take @", takeIndex), nullptr, nullptr);

		timings.Append(GeGetMilliSeconds() - t) iferr_return;

		// createNewTakesTime = createNewTakesTime + GeGetMilliSeconds() - timeStart;
	}
	//ApplicationOutput("Take time " + String::FloatToString(createNewTakesTime) + "    " + 
	//	                String::FloatToString(GeGetMilliSeconds() - loopTimeStart - createNewTakesTime));

	// [FH] Print out the sum of timings and the individual timings.
	ApplicationOutput("Total time: @ sec", maxon::GetSum(timings) * 0.001);
	ApplicationOutput("-------------------------------------------------------------------------");
	for (maxon::Int32 i = 0; i < timings.GetCount(); i++)
		ApplicationOutput("@: @ ms", i, timings[i]);
	
	return maxon::OK;
}

[JohnTerenece]

Thanks for the response Ferdinand, I thought that that would be the case.

John Terenece