How to expose C++ symbols to Python users as a third party?

[ECHekman]

@ferdinand said in how to use the commands from Octane/Redshift's symbols.h:

Hey @RTF,

Thank you for reaching out to us. There is no fully automatic way to do this, as it is up to third parties to expose their APIs to Python. When a vendor does not offer a Python API, you can do two things:

1. Simply define the symbols yourself in your Python code, they are just integer values after all.
2. With mxutils.ImportSymbols you can import symbols from a resource, e.g., a symbols.h file. It is (a part of) the very mechanism we use to automatically expose the symbols from the native Cinema 4D C++ API to the Python API.

Cheers,
Ferdinand

Just to piggyback on this. "it is up to third parties to expose their APIs to Python" How would we at Octane expose the symbols in our c4d_symbol.h?
I found this page: https://developers.maxon.net/docs/py/2024_0_0a/manuals/manual_py_symbols.html
But it doesnt specify how to do this from our end in C++?
Atm we are only able expose certain NodeData headers files in our resource folder.

[ferdinand]

This post is deleted!

[m_adam]

Hi just to let you know I will answers you next week Monday.

Cheers,
Maxime.

[ECHekman]

Definitly interested

[m_adam]

Hi @ECHekman sorry it took me more time than I expected. From my meory octane already have a Python module registered in C++. So I assume you want to add symbols to this python module.

One important aspect is that the Symbol Parser by itself is completly agnostic of the c4d module therefor you can call it with any Python interpreter. This point is really important because this let you parse your C++ header files whenever you want within your build pipeline.

With that's said the Symbol Parser is bundled within the mxutils package which needs the c4d and maxon module.
So in order to have it running with any python intepreter you will need to do the following:

import sys
CI_PROJECT_DIR = r"C:\Users\m_adam.INTERN\Documents\MAXON\gitlab\c4d"

# Build path to import the parser_symbol
maxon_python_path = os.path.join(CI_PROJECT_DIR, "resource", "modules", "python", "libs")
if not os.path.exists(maxon_python_path):
    raise RuntimeError(f"DEBUG: update_python_symbols - Unable to find {maxon_python_path}")

# Get python311 folder where the symbol parser is located
mxutils_path, symbol_parser_path = None, None
for folder in os.listdir(maxon_python_path):
    full_path_folder = os.path.join(maxon_python_path, folder)
    if not os.path.isdir(full_path_folder):
        continue

    # We found the mxutils module containing the symbol parser.
    if os.path.exists(os.path.join(full_path_folder, "mxutils", "symbol_parser")):
        mxutils_path = os.path.join(full_path_folder, "mxutils")
        symbol_parser_path = os.path.join(mxutils_path, "symbol_parser")

if None in (mxutils_path, symbol_parser_path):
    raise ImportError(f"Could not find 'symbol_parser' module path in {maxon_python_path}.")

sys.path.append(mxutils_path)
print(f"DEBUG: Added {mxutils_path} to sys.path.")

# Import the symbol parser and do your stuff.
import symbol_parser
# Do something with it

sys.path.remove(mxutils_path)

Then the Symbol Parser paradigm is that you first parse your data and then the parser contain Scope that contains member (a name and a value).
Therefor once you have parsed your data you can freely output to multiple format if needed. You can also write your own format if you want to.
In this case because you are using mostly the Cinema API and not the Maxon API I re-use the same output as the one we use for the c4d Python package.
This output is implemented in c4d\resource\modules\python\libs\python311\mxutils\symbol_parser\output_classic_api.py. Everything in the Symbol Parser is open source so feel free to look at it.

So to get back to your question find bellow the next code that will parse all the resources files from the Redshift folder and insert them within the "redshift" Python Package. (You usually do not have to do that because Python is parsing automatically the c4d_symbols.h and include such symbols within the c4d Python package this is for the example). With that's said there is basically two ways, one hardcoding the parsed value in your C++ plugin when you register your Python module, the second one using a Python file that will get loaded at the startup of Cinema 4D once your Python module is already loaded and therefor inject symbol into your module.

import os
import c4d # Only used to retrieve various C4D paths used in this example, but not really necessary otherwise

# These imports bellow are not bound to the c4d module therefor they can be used with a standard Python3 interpreter
# But this code is written to be executed in the Script Manager, so you may need to adapt the import statement according to how you run this script.
from mxutils.symbol_parser.extractor import SymbolParser
from mxutils.symbol_parser.output_classic_api import _get_symbol_dict_from_parser


def ParseRedshiftResources() -> SymbolParser:
    # Parse all Redshift ressource files and resolve their values
    rsDir = os.path.join(os.path.dirname(c4d.storage.GeGetStartupApplication()), "Redshift", "res", "description")
    parser = SymbolParser(rsDir,
                            parse_mx_attribute=False,
                            parse_define=True,
                            parse_enum=True,
                            parse_static_const=True)

    # Parse and resolve. Resolve means:
    #  - Transfrom values that depends to others to their actual integer or float values.
    #  - Transform complex values (such as bit mask, arithmetic, etc) into their actual integer or float values.
    # For more information about what is supported take a look at
    # https://developers.maxon.net/docs/py/2025_2_0/manuals/manual_py_symbols.html?highlight=symbol%20parser#symbols-parser-features
    parser.parse_all_files(resolve=True)
    return parser


def OutputPythonFile(parser: SymbolParser):
    def GeneratePythonFileToBeInjected(parser: SymbolParser) -> str:
        """Generate a Python file that will inject all parsed symbols into the 'redshift' Python package when this file get imported.
        This function needs to be called once, most likely during your build pipeline.
        Once you have generated this Python file you should bundle it with your installer and then install it in the targeted Cinema 4D installation.
        See PatchCurrentC4DToInjectParsedSymbol for an example of such deployment
        """
        import tempfile

        tempPath = None
        # Retrieve a sorted dict containing the symbol name as key and it's value as values
        # This will flatten all scopes so an enums called SOMETHING with a value FOO in it will result in a symbol named SOMETHING_FOO
        # If you do not want this behavior feel free to create your own solution, this function is public and declared in
        # c4d\resource\modules\Python\libs\python311\mxutils\symbol_parser\output_classic_api.py
        symbolTable = _get_symbol_dict_from_parser(parser)
        
        with tempfile.NamedTemporaryFile('w', delete=False) as tmpFile:
            tempPath = tmpFile.name
            
            tmpFile.write("import redshift\n")
            
            for name, value in symbolTable.items():
                tmpFile.write(f"redshift.{name} = {value}\n")
            
        
        if not os.path.exists(tempPath):
            raise RunTimeError('Failed to create the Python File to Inject Symbols in "redshift" Python Package') 
        
        return tempPath

    def PatchCurrentC4DToInjectParsedSymbol(fileToBeLoaded: str):
        """This function is going to create a Python file that is going to be called at each startup of Cinema 4D to inject symbols into the "redshift" Python package.
        This is done by placing the previous file in a place where it can be imported by the C4D Python VM.
        And by importing this file by editing the python_init.py file located in the preferences.
        For more information about it please read 
        https://developers.maxon.net/docs/py/2025_2_0/manuals/manual_py_libraries.html?highlight=librarie#executing-code-with-Python-init-py
        """
        import sys
        import shutil
        
        # Pref folder that contains a python_init.py that is laoded once c4d and maxon package is loaded and a libs folder that is part of the sys.path of the c4d Python VM.
        pyTempDirPath = os.path.join(c4d.storage.GeGetStartupWritePath(), f"Python{sys.version_info.major}{sys.version_info.minor}")
    
        # Path to our module that will inject the parsed symbols in the "redshift" Python package
        pyTempRsDirPath = os.path.join(pyTempDirPath, "libs", "rs_symbols")
        # Path to the __init__ file of our module that will be called when we import our module
        # we need to place the file we generated previously in this location.
        pyTempRsFilePath = os.path.join(pyTempDirPath, "libs", "rs_symbols", "__init__.py")
        if not os.path.exists(pyTempRsDirPath):
            os.mkdir(pyTempRsDirPath)
            
        if os.path.exists(pyTempRsFilePath):
            os.remove(pyTempRsFilePath)
            
        shutil.copy2(fileToBeLoaded, pyTempRsFilePath)
        
        # Now that we have our module that will inject symbols within the "redshift" Python package. We need to get this "rs_symbols" module be called at each startup.
        # So we use the python_init.py file to get loaded once all plugins are loaded. Therefor the Redshift plugin is already loaded and have already setup its "redshift" Python package.
        # We will inject a line in this file to load our "rs_symbols" module
        pyTempInitFilePath = os.path.join(pyTempDirPath, "python_init.py")
        
        isImportRsSymbolPresent = False
        initFileExist = os.path.exists(pyTempInitFilePath)
        # Because this file me be already present and already patched or contain other content
        # We should first check if we need to add our import or not
        if initFileExist:
            with open(pyTempInitFilePath, 'r') as f:
                isImportRsSymbolPresent = "import rs_symbols" in f.read()
        
        # prepend our import statement to the file
        if not isImportRsSymbolPresent:
            with open(pyTempInitFilePath, "w+") as f:
                content = f.read()
                f.seek(0, 0)
                f.write('import rs_symbols\n' + content)

    pythonFileGeneratedPath = GeneratePythonFileToBeInjected(parser)
    PatchCurrentC4DToInjectParsedSymbol(pythonFileGeneratedPath)


def OutputCppFile(parser):
    # Retrieve a sorted dict containing the symbol name as key and it's value as values
    # This will flatten all scope so an enums called SOMETHING with a value FOO in it will result in a symbol named SOMETHING_FOO
    # If you do not want this behavior feel free to create your own solution, this function is public and declared in
    # c4d\resource\modules\Python\libs\python311\mxutils\symbol_parser\output_classic_api.py
    symbolTable = _get_symbol_dict_from_parser(parser)
    
    contentToPaste =  'auto mod_rs = lib.CPyImport_ImportModule("redshift");\n'
    contentToPaste += 'auto modDict = lib.CPyModule_GetDict(mod_rs);\n'
        
    for name, value in symbolTable.items():
        contentToPaste += f'\nauto name = lib.CPyUnicode_FromString("{name}");\n'
        contentToPaste += 'if (!name)\n'
        contentToPaste += '  CPyErr_Print();\n'
        contentToPaste += f'maxon::py::CPyRef value = lib.CPyLong_FromInt32({value});\n'
        contentToPaste += 'if (!value)\n'
        contentToPaste += '  CPyErr_Print();\n'
        contentToPaste += 'if (!lib.CPyDict_SetItem(modDict, name, value))\n'
        contentToPaste += '  CPyErr_Print();\n'
                
    return contentToPaste


def main() -> None:
    # Parse the Redshift resources and return the parser that hold the parsed values
    parser = ParseRedshiftResources()

    # First way: Generate a file that patch the current C4D with a Python file that you need to deploy on the installer. You may prefer this option since you can update this file without having to recompile your plugin.
    OutputPythonFile(parser)

    # Second way: Output pseudo C++ code that can be pasted within your C++ plugin after you initialize your custom Python module. You may want to call this whole script before you compile in your build pipeline and put the "cppContent" within a file that will get compiled automatically
    cppContent = OutputCppFile(parser)

if __name__ == '__main__':
    main()

So the system is really modular and being able to run on a regular Python interpreter let you integrate it within your build pipeline to suits your needs.
If you have any questions, please let me know.
Cheers,
Maxime.