custom RenderDocument plugin to render with progress from python. Kind of works but...

pKrime

Hello, I have built a c++ plugin in which I use the RenderDocument function with a render progress hook, pretty much like in the commandline example, and exposed the function to python via

pylib.InitModule("module_name", moduleFunctions, "Module Description")

My custom function takes the document and renderdata as arguments and works as expected: when I send my render command in the script editor or in a python plugin I can see the printout of the render progress.

Only, in some specific cases, the render hangs. For instance, if I render subdivision surfaces in Octane, with full motion blur on:

"Render progress: 0%" is printed, then no further output comes from the render hook
Octane info about exported materials is printed out as usual, but unlike the other times no update about rendering comes out
The GPU is busy, as expected, but no image is written
I see printouts about threads shutting down normally
The GPU stops being used
The VRAM is not emptied
The application freezes

In the debugger, I can see that the internal RenderDocument function doesn't return, but I cannot see what it's really doing as I have no .pdb for the inner functions. I can see that the deassembly code is running a loop of some sort, but I am not proficient in assembly so that's it. There is no GPU nor CPU load

the RenderDocument line is quite standard:

RenderDocument(doc, data, RenderProgressHook, nullptr, bmp, RENDERFLAGS::EXTERNAL | RENDERFLAGS::NODOCUMENTCLONE | RENDERFLAGS::SHOWERRORS, nullptr);

I have tried using no RenderProgressHook, or other combinations of renderflags, tried to run on separate threads both in C++ and Python, and using clones rather than doc and renderdata directly, but the results are always the same.

I have tried to run the very code used in the sdk commandline render, even using commandlinerender.cpp as it is, the results are always the same:

if the render is started inside the c++ plugin, during the C4DPL_COMMANDLINEARGS message, it succeeds
if an Octane render with motion blur and subdivisions is invoked from inside python, either via exposed functions or plugin messages, it freezes

I realise this is kind contrived and I have ventured through a strange path, is there any insight from the community or devs that can help me to fix this issue? Is there any structural barrier that prevents from having a progress hook when python is involved? And in which way the RenderDocument function provided in the python SDK, though without render progress, manages to work around this apparent issue? Does its implementation differ significantly from the one in the commandline example or does it use the C++ SDK too?

Thanks for reading,
Paolo

a_block

Hi Paolo,

this is indeed a bit tricky for us here at MAXON. If I understand correctly, the issue does not happen with our renderers. And here in SDK Team we can not really provide support for external software.

With that said, I'd recommend you try to get in contact with Octane or more specifically Ahmet, who develops the C4D to Octane bridge. Usually he's around in the known C4D forums from time to time, but there may also be some direct contact via Otoy. Your best bet will be to provide him with at least a snippet of your code to reproduce the issue. Then, if he's not able to solve the issue on his end, he could then approach us to further dig down.

Cheers,
Andreas

pKrime

Hi Andreas,

thanks for your reply, I too think that the Octane devs can definetely help with this and I'm working to set up some tests for them to run.

I cannot be sure of which renderers are having problems with this solution, the issue with Octane + Full motion blur is hard to pinpoint itself: is it with the low VRAM? Deferred process? Something with threading that is not working as expected?

But speaking of the Maxon side, can you confirm it is safe to use a renderprogress hook inside a function exposed via pylib? Do you know of someone that has tried it already?

Out of curiosity, how come the python RenderDocument function doesn't do that? Are there any plans to add progress output to the python command? Not necessarily in the callback form: a renderprogress like the one of the commandline render would be useful already

kind regards,
Paolo

a_block

Hi,

I'm sorry for getting back late to you.

We have never done this ourselves nor have we heard of anyone.
The reasons for the difference between C++ and Python API are probably historic. Also back then we had no generic concept for callbacks in Python. Now, with MAXON API and appearance of Delegates and Observables such things will probably handled much nicer in future. Maybe it were also performance considerations.
Many words, for a "I don't know", sorry.

In any case we'll forward this a an idea to our development, so it is at least considered for a future release.

Cheers,
Andreas