Encountered a possible error in the example code in the docs for AssetDataBasesInterface.FindRepository

randymills

I have been working on a bit of code that versions up an asset automatically based on the timestamp of a master asset and I was having a really hard time figuring out how to properly assign the custom database to the new version. I was using the example for AssetDataBasesInterface.FindRepository found here https://developers.maxon.net/docs/py/2024_4_0a/modules/maxon_generated/frameworks/asset/interface/maxon.AssetDataBasesInterface.html#maxon.AssetDataBasesInterface.FindRepository, but I think there may be an error in the code.

I was consistently getting an issubclass error when running the example code:

url: maxon.Url = maxon.Url(r"/Volumes/database")
obj: maxon.ObjectRef = maxon.AssetDataBasesInterface.FindRepository(url)
if obj.IsNullValue():
    raise RuntimeError(f"Could not establish repository for: {url}.")

# Cast #obj to an AssetRepositoryRef. What #obj must be cast to depends on the repository #url
# is representing. But all repositories are of type AssetRepositoryInterface, so a ref for that
# will give access to all common asset repository methods.
repo: maxon.AssetRepositoryRef = maxon.Cast(maxon.AssetRepositoryRef, obj)
if repo.IsNullValue():
    raise RuntimeError(f"Could not establish repository for: {url}.")

After reading a little more about issubclass() I figured out that I needed to define the class to Cast as maxon.AssetRepositoryInterface instead of maxon.AssetRepositoryRef and that seems to work perfectly.

The updated version looks like this and works for me:

url: maxon.Url = maxon.Url(r"/Volumes/database")
obj: maxon.ObjectRef = maxon.AssetDataBasesInterface.FindRepository(url)
if obj.IsNullValue():
    raise RuntimeError(f"Could not establish repository for: {url}.")

# Cast #obj to an AssetRepositoryRef. What #obj must be cast to depends on the repository #url
# is representing. But all repositories are of type AssetRepositoryInterface, so a ref for that
# will give access to all common asset repository methods.
repo: maxon.AssetRepositoryRef = maxon.Cast(maxon.AssetRepositoryInterface, obj)
if repo.IsNullValue():
    raise RuntimeError(f"Could not establish repository for: {url}.")

Thought I'd post here in case anyone else is struggling with this as hard as I've been!

ferdinand

Hey @randymills,

Thank you for reaching out to us. I would not want to rule out that there is either a regression in our API or that I made a mistake when I documented this, but currently it does not look like that to me.

Existing Code

There is for example this thread where I do exactly what is shown in the snippet in the docs, cast an ObjectRef to an AssetRepositoryRef, in that case a sub type. The script still runs just fine for me (2024.5.1, Win 11), and when I inspect the relevant section, the values are also what I would expect them to be, i.e., this:

    obj: maxon.ObjectRef = maxon.AssetDataBasesInterface.FindRepository(url)
    if obj.IsNullValue():
        raise RuntimeError(f"Could not establish repository for: {url}.")

    # #FindRepository returns an ObjectRef, not an AssetRepositoryRef, we must cast it to its "full"
    # form. Specifically, to a reference to a maxon.WatchFolderAssetRepositoryInterface we just 
    # implemented. Yay, casting Python, the dream becomes true :P
    repo: maxon.WatchFolderAssetRepositoryRef = maxon.Cast(maxon.WatchFolderAssetRepositoryRef, obj)
    print(f"{obj = }")
    print(f"{repo = }")

will print that:

obj = maxon.ObjectRef object at 0x1bce66b2720 with the data: net.maxon.assets.repositorytype.watchfolder@0x000001BCAC5814C0<net.maxon.datatype.internedid> net.maxon.assetrepository.skipinsearch : <bool> false
<net.maxon.datatype.internedid> net.maxon.assetrepository.exportonsaveproject : <bool> true
repo = maxon.WatchFolderAssetRepositoryRef object at 0x1bda1573420 with the data: net.maxon.assets.repositorytype.watchfolder@0x000001BCAC5814C0<net.maxon.datatype.internedid> net.maxon.assetrepository.skipinsearch : <bool> false
<net.maxon.datatype.internedid> net.maxon.assetrepository.exportonsaveproject : <bool> true

General Problem that Interfaces and References are not Interchangeable

On top of that comes that your code uses generally a bit flawed logic; which does not mean that there could not be a bug or an exception. Interfaces and references realize a managed memory architecture in our C++ API and are not interchangeable. The interface is the actual object held in memory, to which zero to many references can exist. The lifetime of an object is then managed by reference counting. This is very similar to what managed languages like Python, Java, C# do internally, it is just a bit more on the nose in our API. To briefly highlight the concept at the example of Python:

# This causes Python to allocate the array [0, 1, 2, 3, 4] in its internal object table and then 
# create a reference (the value of id(data)) which is assigned to data.
data: list[int] = [0, 1, 2, 3, 4]
# When we now assign #data to something else, we are just drawing another reference to the actual
# data of #data, #x is the same object as #data.
x: list[int] = data
print(f"{id(data) == id(x) = }") # Will print True, another way to write this would be "data is x".

# Another popular case to demonstrate this are strings, which are also reference managed in most managed
# languages. This means each string is only uniquely held in memory. So, when we allocate "Maxon" twice, 
# there is actually only one object in memory.
a: str = "Maxon"
b: str = "Maxon"
# Although we provided here a string literal in both cases, Python catches us duplicating data and
# only holds the object once in its object table.
print(f"{id(a) == id(b) = }") # Will print True.

# This is a bit deceptive, we are actually not modifying here the string which is referenced by #b,
# because strings are immutable in Python due to the "only stored once logic" shown above. When we
# would modify the actual object, we would also change the value of #a. So, Python does actually copy
# the object referenced by #b and then modifies that copy and assigns a reference to that to #b.
b += "Foo"
print(f"{id(a) == id(b) = }") # Will print False, a and b are not the same object anymore.

id(data) == id(x) = True
id(a) == id(b) = True
id(a) == id(b) = False

Interfaces are similar to the hidden entities living in Python's object table, and references are somewhat equivalent to the frontend data instances a user deals with in Python. We have a manual for Python and C++ about the subject for our API. The C++ manual is slightly better, but both unfortunately do their best to obfuscate the subject with a very technical approach rather than explaining it in laymans terms.

When we really boil all that down, then interfaces are "classes" and references are instances of that classes. That is more an analogy than an actual fact but conveys the major idea that an instances of a class are always meant to be dealt with in the form of references to some interface object.

About your Issue

Because of all that, it strikes me as a bit unlikely that you must cast down a reference to an interface (maxon.AssetDataBasesInterface.FindRepository returns an ObjectRef, not an ObjectInterface, i.e., a reference to some data, not the actual data). Because our managed memory approach does expose that actual raw data (all the SomethingInterface types), you can also technically use the raw object, as all the logic of that type is implemented there. But doing that is very uncommon and you also lack the logic which is provided by references like for example IsEmpty(), IsNullValue(), or GetType() which is not unimportant. References derive from maxon.Data where the these methods provided by references are implemented, interfaces derive from maxon.ObjectInterface and implement the actual functionality of some type, e.g., FindAssets for AssetRepositoryInterface. The methods of an interface then "shine through" on its reference, because the reference is also in inheritance relation to its interface.

When you need more assistance here, or want this generally to be clarified, please share your code. There could be a bug or an inconsistency in our API, or I simply documented something incorrectly. But for me it currently looks more like that something in your code is going wrong.

Cheers,
Ferdinand

randymills

Ferdinand, I stand corrected and apologize for any confusion. Thank you for taking the time to explain that so thoroughly, very helpful! Having examples, like you provided in the documentation here, has been the ONLY way I've been able to navigate my way through the API. I'm quickly finding out the difference between what's included in the Python API vs. the C++ API and learning to work my way around my shortcomings as a coder, so I greatly appreciate your help and your time. After reading your response I have a much clearer understanding of this concept. Thank you!

It does seem that I made a mistake somewhere on my end, since I tried again this morning after reconnecting my custom databases in the Asset Browser and it is working as you described above. I'm not sure what happened yesterday to cause the "issubclass() arg 1 must be a class" error, but it is not reproduceable this morning, and changing the class in maxon.Cast back to "AssetRepositoryRef" now works. I'll continue testing and let you know if I come across this issue again, but I can't seem to reproduce it today and it's working as intended. Would resetting the databases in the Asset Browser make sense to you as to why it may have solved this issue on my end? Could I have stored something other than a subclass in memory somehow?

Again, thank you for your time!

ferdinand

Hey @randymills,

good to hear that there seems to be no issue anymore and there is certainly no need to apologize. We first of all know how easily one can get side tracked in coding and secondly bug reporting is a service to the community and never a mistake even when it turns out to be false alarm. Better having a few false alarms than having bugs in the API or errors in the documentation! Thank your for taking the time.

As to why this happened to you is hard to say. The Maxon API (all the stuff in the maxon package) is not as mature as the Classic API (all the stuff in the c4d package) is in Python. So, I would not rule out some weird corner case 3000 which is buggy. But what I would also not rule out is that you might have shot yourself in the foot with type hinting. You strike me as more experienced user but I have seen less experienced users brick a Cinema 4D instance with type hinting more than once.

# Type hinting is the practice to type annotate code in Python. it is mostly decorative but
# I and some other SDK members tend to write type hinted code as it make clearer examples.

# Typed hinted statement which hints at the fact that result is of type int.
result: int = 1 + 1
# Functionally the same as (unless you use something like Pydantic).
result = 1 + 1

import maxon

# But one can shoot oneself in the foot when getting confused by the type hinting 
# syntax with a colon.

# A correctly type hinted call that states that repo is of type #AssetRepositoryRef 
repo: maxon.AssetRepositoryRef = maxon.Cast(maxon.AssetRepositoryRef, obj)

# We made a little mistake and replaced the colon with an equal sign, maybe because
# type hinting is new to us and the colon feels a bit unnatural to us. What we do here,
# is overwrite  maxon.AssetRepositoryRef, i.e., it is now not the type  AssetRepositoryRef
# anymore but whatever the return value of our expression was. I.e., we do a = b = 1
repo = maxon.AssetRepositoryRef = maxon.Cast(maxon.AssetRepositoryRef, obj)

# Our Cinema 4D instance is now permanently bricked until we restart Cinema 4D because
# we "lost" the type #maxon.AssetRepositoryRef.

Cheers,
Ferdinand

randymills

Thanks Ferdinand and that does seem very plausible given that I am very new to type hinting in my code, and I have actually caught myself swapping the colon for the equals sign already. Learning from you guys every day though! I'm off to break some more sh!t! Thanks so much for your time, will let you know when I get stuck...