TreeView: c4d.DRAGTYPE_FILES

mp5gosu

Hello Café,

what is DRAGTYPE_FILES supposed to do?
The documentation states: Files. The data is a string with the filename.
However, using this flag only has no effect here. I cannot drag anything into the TreeView.

Using DRAGTYPE_FILENAME_OTHER works, but it's horribly slow. It seems like that any file is analyzed to a certain extent.
.c4d files get added very quickly other file types massively lack performance.

Is this a known issue or limitation? Otherwise, I'll file a BL report.

ferdinand

Hi,

I am a bit confused. I know that you are an experienced developer, but given the wording of your question, I have to ask: Are you aware that you are supposed to overwrite TreeViewFunctions.AcceptDragObject()? The dragtype parameter - the flags - is just an indicator for you to interpret the dragobject parameter properly and then return either True or False (you actually have to return a tuple, but you get the gist) in your implementation, to either allow a or disallow a drag operation for the given object.

So It might be useful to provide an code example on what you are doing exactly.

Cheers
zipit

mp5gosu

Hi @zipit,

thank you for the response.
I'm aware of how AcceptDragObject works and probably should have given a bit more detail. (Was a bit late yesterday, hence the short post)

The current code looks like this:
Note: root is actually a simple list that holds objects of type ListItem.

class ListItem looks like this (stripped down version):

class ListItem:
    def __init__(self, filename):
        self.__filename = filename

    @property
    def filename(self):
        return self.__filename

    def __repr__(self):
        return str(self)

    def __str__(self):
        return self.__filename

    def __eq__(self, other):
        return self.__filename == other

The part in TreeViewFunctions:

def AcceptDragObject(self, root, userdata, obj, dragtype, dragobject):
        if dragtype == c4d.DRAGTYPE_FILENAME_OTHER or dragtype == c4d.DRAGTYPE_FILENAME_SCENE:
            filename = path.abspath(path.realpath(dragobject))
            if filename not in root:
                if path.exists(filename.decode("utf8")):
                    root.append(ListItem(filename))
            return c4d.INSERT_AFTER | c4d.INSERT_BEFORE, False

        return c4d.NOTOK, False

As stated in the initial post, it does work like intended but for "foreign" files (Windows, let it be .exe, .dll, . json, .txt, et al) it seems like Cinema analyzes the data chunk of the first bytes of the file contents to distinguish file types (scenes, images and so on). (this is just a assumption)

So much for the background info.
As for the initial question:
In my example dragtype is set to c4d.DRAGTYPE_FILENAME_OTHER since I want to be able to react to certain filetypes and folders.
The docs however do have the flag DRAGTYPE_FILES which - I hoped - would speedup the process by not checking the data chunks. But no luck, it seems like the flag does nothing or works only on different OS.

My question may be a bit misleading - I was hoping for a performance gain since it has to work with thousands of files.

Cheers,
Robert

ferdinand

Hi,

I do not have a TreeView example at hand, so I cannot test if c4d.DRAGTYPE_FILENAME is actually not being sent. Some points:

You are actually not doing what the method is supposed to do. You always return False for the drag operation.
You also modify the root from within AcceptDragObject(), but the place to actually do that is InsertObject(). It seems likely that this does not matter for trees that are drawn manually, but I would not take any chances.

While I would not totally rule out a bug / bad implementation, reading the header of a file (the only reasonable thing to do in this situation), should not really be noticeable. I would more suspect a s***ton of calls to something.

Cheers
zipit

mp5gosu

@zipit said in TreeView: c4d.DRAGTYPE_FILES:

You are actually not doing what the method is supposed to do. You always return False for the drag operation.

That shouldn't matter, because no copy should be made.
The docs say bool: True if copying is allowed to this position, otherwise False. - if I get that right, this is ok.
c4d.NOTOK is here to prevent default behaviour when dragging objects into Cinema 4D, i.e. opening a Cinema 4D scene or images and so on.

You also modify the root from within AcceptDragObject(), but the place to actually do that is InsertObject(). It seems likely that this does not matter for trees that

Ah yes, you are right. When I started implementing the TreeView I actually cerated and inserted the object via InsertObject as usual.
While trying various thing, I found it easier to simply create it here - but yes - you are right. Will revert it. Nice catch.

edit: Just noticed the following:
What ever file I drag into the TreeView, Cinema indeed tries to identify the type.
I observed the debugger output coming up with the following messages:
BaseSound::Identify Error: mediasessionwrongtype [hierarchy.cpp(141)]
majorType: MFMediaType_Audio, subType: MFAudioFormat_MP3, 22050, 1, 8000, 16
BaseSound::Identify Error: Image file:///E:/Downloads/desktop.ini is not of a supported type. [neighborex.cpp(81)]

And some more. Those errors are raised for any file that i unknown to Cinema 4D.
So I still suspect, that C4D tries to identify every filetype I drarg into. That would also explain the performance loss.

edit #2: After some tests, I am now sure that there is some parsing on files. Files that are 0 bytes or known by C4D get added almost in no time.
Garbage or broken files (e.g. bogus.c4d - a renamed text file that with filesize > 0) are analyzed.

m_adam

Hi @mp5gosu what's your final goal? Be able to drag an external file to a TreeView?
If yes then DRAGTYPE_FILENAME_OTHER should be used.

As @zipit pointed AcceptDragObject should only tell if the dragged object is acceptable or not. Since it's called very very frequently during the drag process the slow down can come from there.

InsertObject should be used to insert an object only when the click of the drag process released.

Now regarding the implementation by itself, it's true that some analysis of the file is done so you know if the dragged file is an image (DRAGTYPE_FILENAME_IMAGE), or a scene aka something that could be opened in C4D so a c4d file or fbx (DRAGTYPE_FILENAME_SCENE) or anything else (DRAGTYPE_FILENAME_OTHER).
Normally it shouldn't be an issue since it uses the SceneLoader/BitmapLoader::Identify func to operation (so reading usually only the first bytes, usually 1024). So maybe you have a custom SceneLoader/BitmapLoader that does read more than what he needs and this why you see a slowdown however here I'm not able to see any slowdown even running C4D in debug mode.

Maybe you can share your Code/File.
Cheers,
Maxime.

mp5gosu

Hi Maxime,

thanks for joining in.
@m_adam said in TreeView: c4d.DRAGTYPE_FILES:

Hi @mp5gosu what's your final goal? Be able to drag an external file to a TreeView?

Yes.

If yes then DRAGTYPE_FILENAME_OTHER should be used.

This is my current solution

As @zipit pointed AcceptDragObject should only tell if the dragged object is acceptable or not. Since it's called very very frequently during the drag process the slow down can come from there.

Yes, this is correct. Already reverted, same issue.

InsertObject should be used to insert an object only when the click of the drag process released.

Even with using InsertObject, problem remains - still low performance. See my post above.

Now regarding the implementation by itself, it's true that some analysis of the file is done so you know if the dragged file is an image (DRAGTYPE_FILENAME_IMAGE), or a scene aka something that could be opened in C4D so a c4d file or fbx (DRAGTYPE_FILENAME_SCENE) or anything else (DRAGTYPE_FILENAME_OTHER).

Thanks for the insights. This confirms my observations.

Normally it shouldn't be an issue since it uses the SceneLoader/BitmapLoader::Identify func to operation (so reading usually only the first bytes, usually 1024). So maybe you have a custom SceneLoader/BitmapLoader that does read more than what he needs and this why you see a slowdown however here I'm not able to see any slowdown even running C4D in debug mode.

Maybe you can share your Code/File.

Will do later, when I'm home.

Cheers,
Maxime.

Cheeers,
Robert.

mp5gosu

I set up a minimum plugin to demonstrate the problem.
The issue appears in R19, R20, R21.

Find the Gist here: https://gist.github.com/mp5gosu/e5b86b50ef0c9c4657630c31ce26a44e

Please try this plugin with dragging multiple files of different types and folders into the TreeView.

edit: For my initial problem, I was able to avoid the performance bottleneck since on the plugin I'm working, I'm only interested in c4d files and folders that contains them. Not fast enough though, 600 c4d files take up to 30 seconds - 1 minute. This is way too slow for just getting the file paths.

@m_adam Then what is c4d.DRAGTYPE_FILES for? Just for my knowledge and to answer my very first question.

m_adam

Hi thanks for the file, it was almost looking like mine, and here I can reproduce the issue (i didn't try with more than 4/5 files first) but if I take like 300 I agree it's slow.

The main culprit is not the drag and drops by itself (as suspected) but the insertion of the object, since each object is added one by one (since they are passed one by one into the InsertObject method). And each insertion implies a redraw of the TreeView UI. And if you look in my code sample in the main function if you generate 400 items it takes some time to draw these 400 items (like 20sec). So for each InsertObject the time goes up exponentially.

To counter that I created a temporary array, so in InsertObject I add the object to this temporary array. Then in a Timer (from the GeDialog) I check if there is new data in this temporary array. And if it's the case I add them to our root list, flush the temporary array and redraw only one time. So you still have the 20sec due to the drawing but it's way more usable than before I would say.

Finally, I also get rid of your equality operator since it blocks everything if you drag to drag and drop two time the same file (and looking at your implementation it can only return False if the other type is also a ListItem so I guess you should first try to check the type of input (str or ListItem)).

And here you are

import c4d
from c4d import gui
from c4d import plugins

from os import path

PLUGIN_ID = 1000001 # Test plugin id


class ListItem:
    """
    class ListItem:
    A simple class for storing a single string
    Implements equality operator to be able to easily compare with Filename string BEFORE instantiation,
    see InsertObject()
    """

    def __init__(self, filename):
        self._filename = filename  # The simple String/Filename

    @property
    def filename(self):
        return self._filename

class Tvf(gui.TreeViewFunctions):
    def __init__(self):
        self._draggedData = []

    def GetFirst(self, root, userdata):  # Bare minimum, neccessary to insert ListItems
        if not root:
            return None

        return root[0]

    def GetNext(self, root, userdata, obj):  # Bare minimum, neccessary to insert ListItems
        """
        Get next or None
        Args:
            root (list): The list
            obj (ListItem): The ListItem to get the next element from the list

        Returns: None or next ListItem

        """
        if not root:
            return None

        next_idx = root.index(obj) + 1
        return root[next_idx] if next_idx < len(root) else None

    def AcceptDragObject(self, root, userdata, obj, dragtype, dragobject):
        """
        Checks if the dragged object is allowed.
        Returns NOTOK, to prevent Cinema 4Ds default action when dragobject is not accepted
        """
        if dragtype == c4d.DRAGTYPE_FILENAME_OTHER:
            return c4d.INSERT_AFTER, False

        return c4d.NOTOK, False

    def InsertObject(self, root, userdata, obj, dragtype, dragobject, insertmode, bCopy):
        """
        Creates a ListItem and initializes filename property, then adds it to the list (root)
        """

        # Lock global thread so only our thread is accessing this list (so the timer can't consume data while he add things)
        c4d.threading.GeThreadLock()
        self._draggedData.append(ListItem(dragobject))
        c4d.threading.GeThreadUnlock()


    def GetName(self, root, userdata, obj):
        return obj.filename

    def GetId(self, root, userdata, obj):
        return hash(obj)


class TestDialog(gui.GeDialog):
    IDD_TREEVIEW = 10000
    TV_FILENAME = 11001

    def __init__(self, items):
        self._items = items  # Hold ref to the Filename List
        self._treegui = None  # the TreeViewCustomGui
        self._tv = Tvf()  # instantiate TreeViewFunctions

    def CreateLayout(self):
        self.SetTimer(250)

        settings = c4d.BaseContainer()
        settings.SetBool(c4d.TREEVIEW_OUTSIDE_DROP, True)  # Allows dragging files from outside Cinema 4D
        settings.SetBool(c4d.TREEVIEW_HAS_HEADER, True)

        self._treegui = self.AddCustomGui(self.IDD_TREEVIEW, c4d.CUSTOMGUI_TREEVIEW, "",
                                           c4d.BFH_SCALEFIT | c4d.BFV_SCALEFIT,
                                           300,
                                           300,
                                           settings)

        return True

    def InitValues(self):
        layout_bc = c4d.BaseContainer()
        layout_bc.SetLong(self.TV_FILENAME, c4d.LV_TREE)
        self._treegui.SetLayout(1, layout_bc)

        self._treegui.SetHeaderText(self.TV_FILENAME, "Filename")
        self._treegui.Refresh()

        self._treegui.SetRoot(self._items, self._tv, None)
        return True

    def Timer(self, msg):
        draggedData = self._tv._draggedData
        if not draggedData:
            return

        
        # Lock global thread so only our thread is accessing this list (so the TreeView can't add data while we delete things)
        c4d.threading.GeThreadLock()
        self._tv._draggedData = []
        self._items += draggedData
        c4d.threading.GeThreadUnlock()

        self._treegui.Refresh()


if __name__ == '__main__':
    global dlg
    items = []

    # Create some fake data just to demonstrate how just drawing is slow
    for x in xrange(0):
        items.append(ListItem(str(x)))

    dlg = TestDialog(items)
    dlg.Open(dlgtype=c4d.DLG_TYPE_ASYNC)

Unfortunately for the drawing speed, there is not that much that can't be done since TreeView call a lot of functions recursively (like GetFirst, GetNext) which are in our case implemented in Python. And Python is slow...

And regarding DRAGTYPE_FILES I actually see nowhere in our codebase where it's used, so I would say probably an old relic but I'm asking the development team about it and will come back if I simply overlooked something.

Cheers,
Maxime.

mp5gosu

Hello Maxime,

thanks for that detailed answer. Much appreciated.
I see, drawing slows down things massively. (3minutes vs. a few seconds)
Bad luck for me then.

But it still helped me though, I'm now going for another approach. (The GUI part was just for convenience, so everything is fine)

Thank you and cheers,
Robert