Simulating mouse movement in C4D

[danniccs]

Hi,

I am currently trying to simulate mouse movement using the Python SDK. I can select the tool I want to use, but I have not found a way to simulate the mouse clicking on a screen position, dragging to a different position and letting go of the click. Is this at all possible using the Python API? I have tried calling the ToolData MouseInput() function, but I can only get the tool as a BasePlugin.

Thank you for the help,
Daniel

[ferdinand]

Hello @danniccs,

Thank you for reaching out to us. What you want to do, is not exposed in the SDK, you cannot set the position of the mouse cursor. Moreover, we never expose UI entities in general. So, when you want to control drag events, that is not possible either, as you for example do not have access to the Object Manager dialog.

You can either use a third party library to simulate mouse-user actions. The other route would be to embrace our API and operate on the data (which usually is exposed) rather than the UI. But for that we would have to know what you want to do.

Cheers,
Ferdinand

[danniccs]

Hi Ferdinand, thank you for the answer.

The reason we are trying to simulate mouse input is to create some unit tests in Python for our tool, which inherits from DescriptionToolData (the tool itself is implemented in C++). We override MouseInput() and manipulate geometry within that function, and that is exactly what we need to create a few unit tests for. If we cannot set the position of the mouse cursor or control drag events I will have to think of another way.

The other approach that occurred to me is to create a special case within Message() only for debugging, and run the necessary functions with pre-set mouse positions. Is there any way to get the tool using the Python API in such a way that we can call the Message() override?

Have a great day,
Daniel

[ferdinand]

Hey @danniccs,

Oh, I understand. I am afraid you have chosen a particularly complex case here with no good options. Cinema 4D is simply not built for unit testing its UI (in this manner).

The first thing to get out of the way, is that using Python to unit test C++ code is not a very good idea in the Cinema 4D environment IMHO, as the feature gap is just too large to faithfully test everything you can do in C++. The C++ API also has its own unit testing system.

The added complexity is that you cannot get hold of the actual node/dialog instance for a tool. Remember that a BasePlugin is not a representation of its associated plugin hook: When you implement an ObjectData plugin Ofoo, then there will be a BasePlugin for Ofoo. But that BasePlugin object is neither a BaseObject of type Ofoo that is representing a hook instance of type FooObjectData, nor that hook itself. In the case of DescriptionToolData because of its special nature, I think messages to the BasePlugin are piped through to the tool node (which is inaccessible from the public API). I haven't tested this though, there is a good chance that your messages simply go into the void (as they never reach your plugin). For something like an object, tag, material, etc., all tangible scene elements, this would be much easier, as you could there simply get the entity and then call GeListNode::GetNodeData on it to get direct access to your hook instance, e.g., FooObjectData, or just send messages.

But even when we ignore this already quite messy situation, there is another level of complexity: It will be quite a bit of work to generate meaningful inputs for ToolData::MouseInput that simulate a mous drag and so on. Internally, we test some UI stuff with computer vision (and with that side stepping many problems), but we do not have elaborate tests such as if a drag and drop from x to y does what it should do (i.e., what you want to do). I do not say this to devalue your goals, but to demonstrate that you are on uncharted ground with what you want to do. What we do internally, is that we unit test the logic layer, and in some cases the data layer of a feature. This is also what UnitTestInterface is designed for. This requires meaningful encapsulation.

So, for your case you could have a void DoOperation(PolygonObject* mesh, const Vector& a, const Vector& b, BaseContainer& options) which encapsulates an operation carried out by your tool, which might also be called from separate places. You would then unit test this method and its effect on mesh. I am sure I am not telling you anything new here, but I am just highlighting options.

Cheers,
Ferdinand

[danniccs]

Hi @ferdinand,

Thank you so much for the comprehensive reply! Given all this I don't think it would make sense to keep trying to unit test this particular part of our code with Python. The last approach you highlighted with testing directly on the C++ side seems like the best option, so I'll give that a try. I'll have to see how to integrate this test with the rest of the unit tests (we have a lot and they're all in Python) but hopefully that won't be too much of a problem.

Thanks again for all the help,
Daniel