Data, DataTypes, Python and GUI

ECHekman

For the project I am working I would like to add new data types to the BaseContainer/GeData for a ShaderData and I'm wondering what the best method for storing, displaying, animating and python interop.

The types I want to add are the following vectors:

Int32_2,    Int32_3,    Int32_4
Float_2     Float_4     
Int64_2

I have tried 3 methods so far:

Method 1: CustomDataType from the sdk example:
I implemented this for Int32_3. This one is the most promising so far as it gives me the greatest level of control over the data and GUI.
Drawbacks:

I am not sure how to set/get this data from python or other plugins?
Would much prefer to have proper integrated types like maxon::IntVector32

Method 2: maxon::data types (IntVector32)
I tried IntVector32 and Vector4d for Int32_t and Float_4. The data can be set with the SetMaxonData function or SetData on GeData. This method seems to the most integrated, and these types also exist in python?
Drawbacks:

There seems to be no GUI for these types?
They are not interpolated between keyframes
I dont know how to properly add new GUI for these types as they do not have DTYPE_ value?

Method 3: DTYPE_VECTOR4 / DTYPE_COLORA / DTYPE_VECTOR2
These have GUI.
Drawbacks:

I cannot set or get the data from the BaseContainer. There is no GetVector4. And im not sure how to extract this data from the GeData.
There are no int32 or int64 versions of these

Hopefully you can help me out on navigating this issue.
I would much prefer to use official data types, but im happy to implement my own custom data types. However I am not sure how these properly interop with python and other plugins.

ferdinand

Hello @ECHekman,

Thank you for reaching out to us. Please have a look at Support Procedures: Asking Questions. We understand that it can be difficult to limit yourself to a clear singular question, but we must enforce that, as it becomes otherwise very hard to answer subjects. Your posting violates the singular and clear question rule.

When I read here a bit between the lines, and distill this down to the main question that you want to store vectors of different size and base type in a BaseContainer, your approach of implementing this yourself is possible but unnecessary I would say. You also do not tell us where you vector types come from, because while I do understand implicitly their purpose, they are AFAIK not part of std.

When you do not want to use cinema::Vector and ::Vector4d, you could just use the templates Vec2, Vec3, and Vec4 to just define the vector types you need. There are also plenty using aliases which predefine common cases. cinema::Vector and ::Vector4d are also alias for templates.

Such vector can then then be inserted as maxon::Data in a BaseContainer. In the end all three of your cases are sort of the same, although there exist some extra methods on BaseContainerandGeData` to retrieve some types more easily.

Which leaves us with

I would like to add new data types to the BaseContainer/GeData for a ShaderData and I'm wondering what the best method for storing, displaying, animating and python interop.

You can store pretty much anything in a BaseContainer since you can store there raw memory. Because you can store maxon::Data, this also means you can store a wide range of Maxon API data types without getting hacky with raw memory.
The Python API understands maxon::Data and knows things like maxon::Vec3, but you when you define a new template specialization, it obviously won't know what to do with that. Stuff like IntVector32 is not wrapped for Python.
You usually only need a custom data type and GUI when you want to use that data type as a parameter in a node. E.g., have a shader which has a parameter Enabled which is of type BoolVector3. When you implment a new custom data type, you must always implement a custom GUI (but a new custom GUI can sit on top of an existing data type).
Python won't know what to do with that data type. When you try call MyNode.GetParameter() on that parameter, Python will throw an error.

Bottom line is here, you cannot implement a new data type and have the Python API magically understand it. You could:

Ship you own Python bindings for that data type.
Just implement a custom GUI which makes a Vector4d act like an integer 2, 3, or 4 vector. (That is in fact what the Vector2D custom gui is doing with Vector).

Cheers,
Ferdinand

#include "maxon/datatypebase.h"

maxon::Result<void> Foo()
{
  iferr_scope;

  // A BaseContainer can hold native types as Vector or String, but also generic data in the form
  // of GeData (we can also write raw memory, but I am ignoring this here). GeData can wrap Data
  // which is its Maxon API counter part. And Data can wrap all things that derive from it, which
  // includes the vector templates.

  // Define a custom three component boolean vector (which is of type maxon::Data).
  using BoolVec3 = maxon::Vec3<maxon::Bool, 1>;
  BoolVec3 input (true, true, false);

  // Write it into a container ...
  cinema::BaseContainer bc;
  bc.SetData(0, cinema::GeData(maxon::Data(input)));

  // ... and read it back.
  const maxon::Data& data = bc.GetData(0).GetData();
  BoolVec3 output = data.Get<BoolVec3>() iferr_return;

  // When we wanted to do manual error handling for the data not wrapping the data we
  // think it would wrap, we could do this:
  // 
  // if (data.GetType() != GetDataType<BoolVec3>()) { ... }

  ApplicationOutput("Output: @", output);
  // Output: (true,true,false)

  return maxon::OK;
}

ECHekman

I have two problems with using maxon::data route

For instance I have tried to do int32_3 by using maxon::IntVector32. Which is maxon::Vec3<maxon::Int32, 1>;

However this type does not seem to support interpolating between keyframes
I cant seem to dynamically create GUI for it
For example: This is how i generate UI for the bool checkbox:

const DescID cid = CreateDescID(DescLevel(guiID, DTYPE_BOOL, 0));
BaseContainer bc = GetCustomDataTypeDefault(DTYPE_BOOL);
bc.SetInt32(DESC_CUSTOMGUI, CUSTOMGUI_BOOL);
 bc.SetString(DESC_NAME, name);
bc.SetBool(DESC_SCALEH, scaleH);
bc.SetBool(DESC_FITH, fitH);
bc.SetInt32(DESC_ANIMATE, anim);
description->SetParameter(cid, bc, CreateDescID(DescLevel(groupID)));

How would i do the same for maxon data types?

ferdinand

Hello @ECHekman,

please read my answer I have given above.

You can store pretty much anything in a BaseContainer since you can store there raw memory. Because you can store maxon::Data, this also means you can store a wide range of Maxon API data types without getting hacky with raw memory.

The Python API understands maxon::Data and knows things like maxon::Vec3, but you when you define a new template specialization, it obviously won't know what to do with that. Stuff like IntVector32 is not wrapped for Python.

You usually only need a custom data type and GUI when you want to use that data type as a parameter in a node. E.g., have a shader which has a parameter Enabled which is of type BoolVector3. When you implement a new custom data type, you must always implement a custom GUI (but a new custom GUI can sit on top of an existing data type).

Python won't know what to do with that data type. When you try call MyNode.GetParameter() on that parameter, Python will throw an error.

Bottom line is here, you cannot implement a new data type and have the Python API magically understand it. You could:

Ship you own Python bindings for that data type.

Just implement a custom GUI which makes a Vector4d act like an integer 2, 3, or 4 vector. (That is in fact what the Vector2D custom gui is doing with Vector).

You neither can just have magically a GUI manifest for some data type, nor can you use maxon::Data as a parameter type directly. As I stated in my previous answer, when you want to use your new data type(s) as parameter(s) types - opposed to just storing them in the data container of the node - you will have to implement a custom data type and GUI for them. But the Python API will then not be able to deal with such data types (unless you make them decomposable into existing data types - which would be possible in your case).

Cheers,
Ferdinand

ECHekman

Just to clear something up that you wrote: maxon::IntVector32 is not a new template specialization.
And are you sure maxon::IntVector32 is not wrapped in python. It is in the python sdk?
https://developers.maxon.net/docs/py/2025_0_0/modules/maxon_generated/datatypes/vec_col_mat/maxon.IntVector32.html?highlight=intvector32#module-maxon.IntVector32
Or am I missing something here?

I have the UI working for the pre-existing maxon::IntVector32 data type. The only problem with this method is that when i animate it, it does not interpolate between multiple keyframes - it just jumps from one to the other.

The problem im trying to solve is that Octane supports these other types (int3 float4 long2 etc). But from what I now learned, and reading your answers, I will either need to chose between:

Use Custom Data Type - dropping python support
Use maxon::IntVector32 - dropping keyframe interpolation support
Cast everything from Vector4d to int vectors and in64 vectors - possible data loss and clunkyness by having to set int values with floats in python

Btw, I recently got backstage access, would it be better to ask these questions there?

ferdinand

Hey @ECHekman,

you are right, IntVector32 has been wrapped by Maxime, my bad. Regarding being a new specialization or not, I guess that depends a bit on the perspective. IntVector32 is an alias for a specialization of Vec3.

And what I was hinting at, is that when you define your own specialization, as I did above for my bool vector example BoolVec3, the Python API won't automatically know how to deal with it (although it knows what a Vec3 is). But you can wrap a data type relatively easily with pure Python code yourself (check out {C4D}\resource\modules\python\libs\python311\maxon and the files vec, vec4, and vector). To then use that type you would have to either put that binding into each script, import the binding from a module in each script, or make it so that your plugin puts it into \python311\_maxon_init so that it is automatically exposed in the maxon package (please be careful with patching a users __init__.py, as it could be customized, so you cannot blindly overwrite that file).

Regarding the rest, I already lined out your options above. Maybe I am still misunderstanding you, but you cannot use a Maxon datatype as a parameter, nor can you implement your own Cinema datatype and have the Python API understand it (1). As lined out above, the best option for you is likely writing a custom GUI which emulates 2, 3, or 4 component integer vector type and sits on top of the existing data type Vector4D.

Regarding your concerns about precsion: The error in a Float64 (the underlying component type of Vector/DYTPE_VECTOR) becomes larger than 1 when the value exceeds 2^53. That is outside of the interval [-2^31, 2^31 - 1] of Int32.

Regarding the 'UI working for maxon::IntVector32': I can only speculate what you are doing there, my assumption would be that you probably have some form of dynamic description and shove that IntVector32 into the data container of the node. Feel free to expore this route, but that is not supported. (Cinema API) Descriptions must be composed out of Cinema API datatypes. The interpolation issues you encounter are like tied to how CKey::Set/GetValue and CKey::Set/GetData work and the API then getting confused about the datatype in the data container of the scene element which does not conform to a registered DTYPE_ (or on point, it likely treats your values as data although they are values).

To stress it gain, a custom GUI which wraps arround Vector4D is porbably what you want to do here. If this is not a fit, please share concrete code and a concrete usage scneario, as we won't get anywhere otherwise.

Cheers,
Ferdinand

(1) With the exception of data types which are decomposable into channels, e.g., what DTYPE_VECTOR does and many datatypes in Redshift do. So, you would then end up with a custom datatype DTYPE_FOO and the Python API would still have no clue what to with it when the user calls myNode[c4d.FOO_PARAMETER] and then raise an error. But when your data type is decomposable into known types (three DTYPE_LONG in your case), the user could call myNode[c4d.FOO_PARAMETER, c4d.VECTOR_X] to directly access the x-component of it.

ECHekman

Ok thanks for the help Ferdinand. I will take the road of least resistance and just build on top of float64 vectors