Change textures path from GetAllAssetNew in the new node editor

[Tng]

Hello!

First of all a disclaimer, I'm a beginner in python and coding in general, so if i'm doing things the wrong way don't hesitate to point me to the relevant documentation

For context, this script is aimed to rassemble all the used textures of the scene into a folder, either on a local disk or network storage in a studio environment. The goal is to support all shader types, and for the moment i have Standard, Octane and RS Xpresso thanks to some code from Aoktar and r_gigante.

My problem lies with the new Node Editor and material system which confuses me a lot.
I think i have read all the topics talking about textures paths, i even tried to understand the RS Api made by DunHouGo but i couldn't find anything that ressemble the workflow i'm using for the other types of materials.

I think i will also have a problem down the line with my method to compare filename if the texture is located in the asset browser, so if you have any pointers towards that would be incredible!

Here is the code :

import pipelineLib
import os
import shutil
import c4d
import filecmp
import maxon

def get_tex_path_local():
    path = d:\\02_3D\\c4d_cache
  
    return local_path


def sync_folders(src_folder, dst_folder):  # FILE SYNC
    if not os.path.exists(dst_folder):
        os.makedirs(dst_folder)
        print(f"The destination folder '{dst_folder}' was created")

    src_files = set(os.listdir(src_folder))
    dst_files = set(os.listdir(dst_folder))

    # Find files that are in src_folder but not in dst_folder
    new_files = src_files - dst_files
    for filename in new_files:
        src_file = os.path.join(src_folder, filename)
        dst_file = os.path.join(dst_folder, filename)
        with open(src_file, 'rb') as fsrc:
            with open(dst_file, 'wb') as fdst:
                fdst.write(fsrc.read())

    # Find files that are in both src_folder and dst_folder
    common_files = src_files & dst_files
    for filename in common_files:
        src_file = os.path.join(src_folder, filename)
        dst_file = os.path.join(dst_folder, filename)
        if not filecmp.cmp(src_file, dst_file):
            with open(src_file, 'rb') as fsrc:
                with open(dst_file, 'wb') as fdst:
                    fdst.write(fsrc.read())

def get_doc():
    doc = c4d.documents.GetActiveDocument()
    return doc


def get_materials_dict():
    if not hasattr(get_materials_dict, 'textures'):
        doc = get_doc()
        textures = list()
        c4d.documents.GetAllAssetsNew(doc, False, "", c4d.ASSETDATA_FLAG_TEXTURESONLY, textures)
        get_materials_dict.textures = textures
    
    for t in textures:
        print("DICT:", t)
    return get_materials_dict.textures

def set_redshift_node_path_local():
    output_path = get_tex_path_local() #Get the new path
    textures = get_materials_dict()  # GetAllAssetsNew
    textures_rs = [t for t in textures if 'texturesampler' in str(t['nodePath'])] # Clean the dictionary 
    for t in textures_rs:
        textureOwner = t["owner"]
        filename = t["filename"]
        head, tail = os.path.split(filename)  # Filename extract
        new_filename = os.path.join(output_path, tail)  # Filename merge
        if not os.path.exists(new_filename) and "\\images\\" not in filename:  # Verify if the file exist in the folder
            shutil.copy2(filename, new_filename)
        if filename == new_filename:
            print("File already exist in local cache")  # Identical file check
            ...
        elif "d:\\02_3D\\c4d_cache" in filename: 
            print("File already exist in local cache")  # Folder path check
            ...
        elif "\\images\\" in filename:  # Skip Roto/Plate images on the server
            print("File skipped")
            ...
        else:
            try:  # Set the new path, this is where it's not working
                print("try") 
                textureOwner[maxon.Id("com.redshift3d.redshift4c4d.nodes.core.texturesampler.tex0")] = new_filename
            except:
                print("Not supported, use other scripts")
        # notify Cinema about the changes
        c4d.EventAdd()

This is what the console outputs :

# Verify if the project is connected to the pipe 
tpl : \\my\network\folder 
# Local cache output path resolved if the pipe exist, otherwise print error and return
tpl : d:\02_3D\c4d_cache\motion_dev\tex

DICT: {'filename': 'C:\\Users\\myname\\Downloads\\camellia-4881662.jpg', 'assetname': 'C:\\Users\\myname\\Downloads\\camellia-4881662.jpg', 'channelId': 3, 'netRequestOnDemand': True, 'owner': <c4d.Material object called Mat/Mat with ID 5703 at 3191915188736>, 'exists': True, 'paramId': -1, 'nodePath': 'texturesampler@A$nLrnAxFRmq2S7wjxLCrZ', 'nodeSpace': 'com.redshift3d.redshift4c4d.class.nodespace'}
DICT: {'filename': 'C:\\Users\\myname\\Pictures\\images.jpg', 'assetname': 'C:\\Users\\myname\\Pictures\\images.jpg', 'channelId': 3, 'netRequestOnDemand': True, 'owner': <c4d.Material object called Mat.1/Mat with ID 5703 at 3191915354112>, 'exists': True, 'paramId': -1, 'nodePath': 'texturesampler@Bsjai0RbH5Gt2sgzcSnRtl', 'nodeSpace': 'com.redshift3d.redshift4c4d.class.nodespace'}
try
try

But unfortunatly the path doesn't change, i know it's wrong but i cannot wrap my head around this logic for the moment
Thank you in advance for any help you can provide!

[ferdinand]

Hello @Tng,

Welcome to the forum and thank you for reaching out to us. The reason why your code is not working is because you have there a Redshift node material, and the owner passed to you is a c4d.Material. The instruction

textureOwner[maxon.Id("com.redshift3d.redshift4c4d.nodes.core.texturesampler.tex0")] = new_filen

Is therefore meaningless, as you are here trying to use a maxon ID as an identifier for a (non-eixsting) classic API parameter on the material. The Python interpreter should complain about it with the following error message:

TypeError: GeListNode_mp_subscript expected Description identifier, not Id

Doing what you want to do here, replace a texture reference in things that use bitmaps, will require a bit more work for node materials, as you are only given the node material and not the actual port to which the texture value must be written (which is not possible out of context because node graphs use transactions).

Find below a simple example which sketches out the major steps to take. Please understand that it is indeed an example and not a finished solution. Supporting all possible node spaces and all possible nodes for Redshift could be quite a bit of work. I also condensed down your file copying code to two lines of more illustrative nature than trying to emulate what you did there.

Helpful might here also be the Python Nodes API examples.

Cheers,
Ferdinand

Result

Code

"""Demonstrates the inner makeup of classic API MSG_GETALLASSETS asset data and how to access
referencing entities in case of node materials.
"""

import c4d
import os
import maxon
import typing
import shutil

doc: c4d.documents.BaseDocument # The active document.

def MoveTextureAssets(assetData: list[dict], path: str) -> int:
    """Copies assets in #assetData to the new location #path and attempts to update all referencing
    node material and shader owners.
    """
    # Make sure the target path does exist.
    if not os.path.exists(path):
        raise IOError(f"Target path '{path}' does not exist.")

    # Asset data can come in many forms, but the important distinction for us here is that there can
    # be node material and bitmap shader assets.
     
    # Asset data for a node material node using an Asset API asset.
    # {'filename': 'asset:///file_45767ea1e1e47dff~.jpg', 
    #  'assetname': 'assetdb:///Textures/Surfaces/Concrete/concrete wall_sq_bump.jpg', 
    #  'channelId': 3, 
    #  'netRequestOnDemand': True, 
    #  'owner': <c4d.Material object called Node/Mat with ID 5703 at 3146805000832>, 
    #  'exists': True, 
    #  'paramId': -1, 
    #  'nodePath': 'texturesampler@QkFq7b6VABSsvy3ddKE3bp', 
    #  'nodeSpace': 'com.redshift3d.redshift4c4d.class.nodespace'}

    # Asset data for a bitmap shader using a local file asset.
    # {'filename': 'E:\\misc\\particles.png', 
    #  'assetname': 'E:\\misc\\particles.png', 
    #  'channelId': 0, 
    #  'netRequestOnDemand': True, 
    #  'owner': <c4d.BaseShader object called Bitmap/Bitmap with ID 5833 at 3146805004480>, 
    #  'exists': True, 
    #  'paramId': 1000, 
    #  'nodePath': '', 
    #  'nodeSpace': ''}

    # Iterate over the data and get the relevant fields for each item.
    for item in assetData:
        assetExists: bool = item.get("exists", False)
        nodePath: str = item.get("nodePath", "")
        nodeSpace: str = item.get("nodeSpace", "")
        oldPath: str = item.get("filename", "")
        owner: typing.Optional[c4d.BaseList2D] = item.get("owner", None)
        paramId: int = item.get("paramId", c4d.NOTOK)
        

        # We skip over all non-existing and Asset API assets, i.e., files which are in
        # the "asset" scheme. We could also localize them, but that would be up to you to do.
        if not assetExists or oldPath.startswith("asset:"):
            print ("Skipping over non-existing or Asset API asset.")
            continue
        
        # Copy the file when it has not already been copied.
        newPath: str = os.path.join(path, os.path.split(oldPath)[1])
        if not os.path.exists(newPath):
            shutil.copy(oldPath, newPath)

        # There is no error-proof indication for it, but this is very likely a bitmap shader asset. 
        # We could test for something like Xbitmap, but that would exclude all non-standard
        # render engines that do not re-use that shader, so a rough estimate it is. Here it is
        # simple parameter access.
        if isinstance(owner, c4d.BaseShader) and paramId != c4d.NOTOK and nodePath == "":
            owner[paramId] = newPath

        # This is a node material asset, here the owner is nothing more than the material which owns
        # the node graph hook where we find our data.
        if isinstance(owner, c4d.BaseMaterial) and nodePath != "" and nodeSpace != "":
            # Get the node material associated with the material and get the graph for the node 
            # space indicated in the asset data.
            nodeMaterial: c4d.NodeMaterial = owner.GetNodeMaterialReference()
            if not nodeMaterial:
                raise MemoryError(f"Cannot access node material of material.")

            graph: maxon.GraphModelInterface = nodeMaterial.GetGraph(nodeSpace)
            if graph.IsNullValue():
                raise RuntimeError(f"Invalid node space for {owner}: {nodeSpace}")

            # Start a graph transaction and get the node for the node path given in the asset data.
            with graph.BeginTransaction() as transaction:
                node: maxon.GraphNode = graph.GetNode(maxon.NodePath(nodePath))
                if node.IsNullValue():
                    raise RuntimeError(f"Could not retrieve target node {nodePath} in {graph}.")

                # Redshift only gives us the "true" node with the node path and not directly the 
                # port. In Python it is currently also not possible to get the node Asset ID to
                # determine the node type. So, we use the node Id and some string wrangling to
                # make an educated guess. Because Redshift (other than the standard renderer for
                # example) does not give us the full path to the port with "nodePath", we must
                # hardcode every node type that can reference a texture (I am not sure if there is
                # more than one for RS). 
                if (nodeSpace == "com.redshift3d.redshift4c4d.class.nodespace" and 
                    node.GetId().ToString().split("@")[0] == "texturesampler"):
                    pathPort: maxon.GraphNode = node.GetInputs().FindChild(
                        "com.redshift3d.redshift4c4d.nodes.core.texturesampler.tex0").FindChild(
                            "path")
                    if pathPort.IsNullValue():
                        print (f"Stepping over Redshift non-Texture node target node.")
                        continue
                    pathPort.SetDefaultValue(newPath)
                # Here you would have to implement other node spaces as for example the standard 
                # space, Arnold, etc.
                else:
                    print (f"Unsupported node space {nodeSpace}.")
                    continue
                    
                transaction.Commit()

    
def main() -> None:
    """Runs the example.
    """
    assetData: list[dict] = []
    c4d.documents.GetAllAssetsNew(doc, False, "", c4d.ASSETDATA_FLAG_TEXTURESONLY, assetData)
    MoveTextureAssets(assetData, "e:\\temp")

if __name__ == "__main__":
    main()

[Tng]

Thank you very much for this example it is very helpful! I have integrated it into my code and it works so well

I've combed through your code and comments, i think i get most of it!! I will try to integrate other nodes spaces and post the result if the code can be of use to anyone

One question regarding Assets from the asset browser, in the case of a random machine that is rendering via command-line, is the Asset stored in the Project File or will it be downloaded if's missing ?

Thanks again for the help ferdinand!!

[ferdinand]

Hello @Tng,

Please excuse the very long waiting time, I simply overlooked the question in your answer.

One question regarding Assets from the asset browser, in the case of a random machine that is rendering via command-line, is the Asset stored in the Project File or will it be downloaded if's missing ?

1. Assets will be automatically downloaded when they are accessed and not already localized. The type maxon.Url hides all that complexity away with its different URL schemes. The Asset API manual provides an example for how to manually download an asset. But doing that is not possible in the Python API, here one must always rely on the automated asset access mechanisms.
2. Assets can be stored in project files (via BaseDocument.GetSceneRepository) but are usually not, and instead reside in the user preferences directory bound user preferences repository. Once a remote asset has been accessed, a copy of its primary data, e.g., the texture, the model, etc., resides as a cache in the user preferences of that Cinema 4D installation. That cache will only be updated when the primary or secondary (i.e., metadata) of that asset change.

Cheers,
Ferdinand

PS: I have closed this thread due to its age. Please feel free to reopen it when you have more questions.