Asset Browser: Add Watch Folder

a_block

Hi,

I was pretty excited, when the Watch Folders for the Asset Browser got introduced.

But trying to find a function or command to add a Watch Folder from Python API, I have to admit, I'm a bit lost. Is this possible?

Cheers,
Andreas

ferdinand

Hey @a_block,

Thank you for reaching out to us and that is a good question. I haven't had a closer look at the Python Asset API since S26. In C++, you must use WatchFolderAssetRepositoryInterface to do this the "right way". This type has not been wrapped yet for Python. But my hunch would be that you can hack your way to success with the user preferences in Python, similar how to one can also add databases like that (although that is not necessary in anymore, as the required interfaces have been wrapped).

I will have a look and see what I can do, but it will probably take me some time.

Cheers,
Ferdinand

ferdinand

Hello @a_block,

so I gave it a spin, and the stuff I was thinking of has been removed by Tilo with the recent watch folder updates. We will provide native watch folder support in an upcoming release.

With that being said and knowing that it being provided with 2023.2 or later will throw a wrench into things for you, there are two ways how you can wiggle your way to success here.

Simply implement watch folder repositories yourself. It is not that hard, even I can do it The ::RemoveWatchFolder method does not work in my wrapper, don't ask me why. We will have to wait for Maxime's 'proper' version there. But that does not really prevent us from unmounting watch folders, as they are just a database like any other.
The other way would be getting creative with relative watch folders associated with a document. When you dig deep enough, you could probably abuse this. But since (1.) is available, why would we want to do this?

Find an example for option one and hints for option two below.

Cheers,
Ferdinand

The result:
Screenshot 2022-12-05 at 15.51.14.png

asset.GetUrl() = file:///Users/f_hoppe/Documents/plugins/2023/sdk/frameworks/settings/sourceprocessor/pythonsp/macos/lib/python3.9/test/xmltestdata/c14n-20/c14nTrim.xml
asset.GetUrl() = file:///Users/f_hoppe/Documents/plugins/S24/cinema4d_py_sdk_extended-master/scripts/04_3d_concepts/scene_elements/scene_management/basedocument_loops_marker_r17.py
asset.GetUrl() = file:///Users/f_hoppe/Documents/plugins/2023/sdk/frameworks/settings/sourceprocessor/pythonsp/macos/lib/python3.9/test/decimaltestdata/ddInvert.decTest
asset.GetUrl() = file:///Users/f_hoppe/Documents/plugins/2023/sdk/frameworks/settings/sourceprocessor/pythonsp/macos/lib/python3.9/unittest/test/testmock/__init__.py
asset.GetUrl() = file:///Users/f_hoppe/Documents/plugins/2023/sdk/frameworks/settings/sourceprocessor/pythonsp/macos/lib/python3.9/test/cjkencodings/gb18030-utf8.txt
asset.GetUrl() = file:///Users/f_hoppe/Documents/plugins/2023/sdk/build/cinema_hybrid.framework.build/Debug/cinema_hybrid.framework.build/Script-A0C6A0EC1A00000000690000.sh
doc[c4d.DOCUMENT_MOUNTWATCHFOLDER] = 1
doc[c4d.DOCUMENT_RELATIVEWATCHFOLDER] = 'tex'

The code:

"""Provides an example for handling watch folders in Python as of 2023.0.0.

THIS IS A WORKAROUND. We will add the necessary wrappers with an upcoming release, but in the mean
time one could use this as this is close enough to the wrapper Maxime will provide.

I am using here 2023.0.0 API calls, not just for the watch folder stuff. Just pointing this out 
because `asset_watchfolder.h` is technically since S26 around, but was not fully functional then.
"""
__version__ = "2023.0.0"
__copyright__ = "Maxon Computer GmbH"

import os

import c4d
import maxon

#  --- Start of wrapper code -----------------------------------------------------------------------

# Some wrapper stuff.
from maxon.consts import MAXON_REFERENCE_NORMAL
from maxon.interface import MAXON_INTERFACE, MAXON_STATICMETHOD
from maxon.reference import MAXON_REFERENCE

# Wrapping WatchFolderAssetRepositoryInterface
@MAXON_INTERFACE(MAXON_REFERENCE_NORMAL, "net.maxon.interface.watchfolderassetrepository")
class WatchFolderAssetRepositoryInterface(maxon.AssetRepositoryInterface):
    """Represents a repository for a watch folder containing all watched assets.

    This is an AssetRepositoryInterface as any other, you can do all the things with it you can
    do with other AssetRepositoryInterface instances.
    """
    @staticmethod
    @MAXON_STATICMETHOD("net.maxon.interface.watchfolderassetrepository.AddWatchFolder")
    def AddWatchFolder(url: maxon.Url, active: bool) -> None:
        """Adds #url as a watch folder to the running Cinema 4D instance.

        Args:
            url: The url to add.
            active: If True, the repository will be shown in the Asset Browser and contribute
             its assets to the discoverable assets. If #False, the database will only be mounted
             but not activated.
        """
        pass

    @staticmethod
    @MAXON_STATICMETHOD("net.maxon.interface.watchfolderassetrepository.RemoveWatchFolder")
    def RemoveWatchFolder(repo: 'WatchFolderAssetRepositoryRef') -> None:
        """Does not work.
        """
        pass

# Wrapping the Ref for a WatchFolderAssetRepositoryInterface
@MAXON_REFERENCE(WatchFolderAssetRepositoryInterface)
class WatchFolderAssetRepositoryRef(
    WatchFolderAssetRepositoryInterface, maxon.AssetRepositoryRef, maxon.Data):
    """References a `maxon.WatchFolderAssetRepositoryInterface` object.
    """
    def __new__(cls, *args):
        return object.__new__(cls)

# Add both types to the maxon namespace, this is more form than necessity.
maxon.WatchFolderAssetRepositoryInterface = WatchFolderAssetRepositoryInterface
maxon.WatchFolderAssetRepositoryRef = WatchFolderAssetRepositoryRef

# --- End of wrapper code --------------------------------------------------------------------------

def GetWatchFolderRepo(
        url: maxon.Url, forceCreate: bool = False) -> maxon.WatchFolderAssetRepositoryRef:
    """Returns the repository for the given watch folder #url.

    This is a helper function to hide away some boiler plate code on calling ::AddWatchFolder. As
    far as I can see, there are is no helper method for this even in the C++ API.

    Args:
        url: The #url of the repository to retrieve.
        forceCreate: If true, the watch folder will be mounted when not already mounted. Otherwise,
         only an existing repository is retrieved.

    Returns:
        The repository reference for #url.
    """
    # Assert that #url is a URL for a directory.
    if not isinstance(url, maxon.Url):
        raise TypeError(f"{url = }")
    
    # UrlInterface::IoDetect is a more maxonic way to do what I am doing here, but ehhhhhh :)
    sysPath: str = url.GetSystemPath()
    if not os.path.exists(sysPath):
        raise IOError(f"The path '{sysPath}' is not accessible.")
    if not os.path.isdir(sysPath):
        raise IOError(f"The path '{sysPath}' is not a directory.")

    # Mount the watch folder when so indicated, a database cannot be added twice with AddWatchFolder,
    # so no checks are necessary to test if it has already been mounted.
    if forceCreate:
        maxon.WatchFolderAssetRepositoryInterface.AddWatchFolder(url, True)

    # Wait for all databases to load and then try to retrieve the repository for #url.
    if not maxon.AssetDataBasesInterface.WaitForDatabaseLoading():
        raise RuntimeError("Could not load asset databases.")

    obj: maxon.ObjectRef = maxon.AssetDataBasesInterface.FindRepository(url)
    if obj.IsNullValue():
        raise RuntimeError(f"Could not establish repository for: {url}.")

    # #FindRepository returns an ObjectRef, not an AssetRepositoryRef, we must cast it to its "full"
    # form. Specifically, to a reference to a maxon.WatchFolderAssetRepositoryInterface we just 
    # implemented. Yay, casting Python, the dream becomes true :P
    repo: maxon.WatchFolderAssetRepositoryRef = maxon.Cast(maxon.WatchFolderAssetRepositoryRef, obj)
    if repo.IsNullValue():
        raise RuntimeError(f"Could not establish repository for: {url}.")

    return repo

def UnmountDatabase(url: maxon.Url) -> None:
    """Unmounts the database with the given URL.

    .RemoveWatchFolder() does not work in this implementation, don't ask me why. It will be fixed when
    Maxime implements it :P. But removing a watch folder manually is not that hard, since it is just
    a user database.

    You could also do the other things which I showed in the Asset API manuals, e.g., keep the 
    database mounted, but deactivate it.

    Args:
        url: The URL of the database to unmount, could be a watch folder url.
    """
    # Assert that #url is a URL for a directory.
    if not isinstance(url, maxon.Url):
        raise TypeError(f"{url = }")
    
    sysPath: str = url.GetSystemPath()
    if not os.path.exists(sysPath):
        raise IOError(f"The path '{sysPath}' is not accessible.")
    if not os.path.isdir(sysPath):
        raise IOError(f"The path '{sysPath}' is not a directory.")

    if not maxon.AssetDataBasesInterface.WaitForDatabaseLoading():
        raise RuntimeError("Could not load asset databases.")

    # Build the set of all user databases which does not contain the database with the root #url.
    collection: list[maxon.AssetDatabaseStruct] = [
        db for db in maxon.AssetDataBasesInterface.GetDatabases()
        if db._dbUrl != url
    ]

    # And mount this new set, we could also check here if something did actually change to avoid
    # overhead, but I am lazy.
    maxon.AssetDataBasesInterface.SetDatabases(collection)
    if not maxon.AssetDataBasesInterface.WaitForDatabaseLoading():
        raise RuntimeError("Could not load asset databases.")

doc: c4d.documents.BaseDocument
    
def main():
    """Runs the example.
    """
    # The folder to mount, you should obviously provide here a path of your own.
    url: maxon.UrlInterface = maxon.Url(r"/Users/f_hoppe/Documents/plugins")

    # Add the repository for a watch folder at #url, we use here our little helper function to hide
    # away a bit of boiler plate code.
    repo: maxon.WatchFolderAssetRepositoryRef = GetWatchFolderRepo(url, forceCreate=True)
    if not repo:
        raise RuntimeError(f"Could not establish watch folder for: {url}")

    # Find all assets in #repo, see the Asset API examples for more elaborate search operations. As
    # always, all assets of #repo will also be contained in the user prefs repo.
    assetCollection: list[maxon.AssetDescriptionInterface] = []
    repo.FindAssets(maxon.Id(), maxon.Id(), maxon.Id(), maxon.ASSET_FIND_MODE.LATEST, assetCollection)

    # Iterate over the first five or less assets in #repo.
    for asset in assetCollection[:6]:
        print (f"{asset.GetUrl() = }")

    # And remove the database again.
    # UnmountDatabase(url)

    # --- Some BaseDocument shenanigans ------------------------------------------------------------

    # I did not flesh this one out, as it is very hack and probably not a good idea. These are the
    # settings of the watch folders associated with a document. By default this is just the common
    # Cinema 4D 'tex' folder, but I think you can have more than one of them. There is also stuff
    # in the world prefs. You would have to poke around a bit here and there, but I would not
    # recommend doing so.

    print (f"{doc[c4d.DOCUMENT_MOUNTWATCHFOLDER] = }")
    print (f"{doc[c4d.DOCUMENT_RELATIVEWATCHFOLDER] = }")


if __name__ == "__main__":
    main()

a_block

Thanks, Ferdinand. I'll take a closer look as soon as I find the time.