Rendering and Threads [SOLVED]

Helper

On 22/10/2014 at 08:01, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   R14
Platform:   Windows ;
Language(s) :     C++ ;

---------
Hi Folks,

I'm trying to get some simple renderings via a separate thread. The setup is fairly simple, there's a scene object (with plugin id) which has a C4DThread object at class level. On the click of a command, I'm sending a MultipassBitmap reference to the thread object for the thread to draw into, setting up the render settings container, and then calling the Thread.Start(). Using some simple GePrint()'s I can see that the thread happily sits in the background doing it's thing while leaving C4D's main thread alone (and free to work with).

But all my renderings done from inside the custom thread are returning black/blank images. I can use the same setup and run it from Cinema's main thread and it renders fine. Here's the setup I'm using (shortened) :

BaseDocument *SceneDocument = GetActiveDocument();
BaseContainer RenderSettings;
// set RenderSettings container elements here
 
MultipassBitmap *RenMPB = Alloc(..);
RenMPB->AddLayer(RenMPB->GetLayerNum(0));
 
if(RenderDocument(SceneDocument,RenderSettings,NULL,NULL,RenMPB->GetLayerNum(1),RENDERFLAGS_EXTERNAL|RENDERFLAGS_PREVIEWRENDER,NULL) == RENDERRESULT_OK)
{
    GePrint("Render successful..");
}
 
// Am wanting to render to layer index 1 of the MultipassBitmap. 
// Works and prints fine from Cinema's main thread (picture displays fine).
// 'Works' and prints successful message from custom thread, but the image is blank.
// The picture viewer is called once the thread has stopped.

Is there something about the above setup that thread's don't like? Are the render flags alright to use in a thread context? Something else I may not be doing, or aren't aware of?

Cheers,

WP.

Helper

On 22/10/2014 at 08:17, xxxxxxxx wrote:

I "guess" that the main thread can prepare data "like Ray Objects" so when you render, you can "see" the scene, while the other thread can't prepare data, and thus rendering an empty scene //this is just a guess and it may be wrong

Helper

On 23/10/2014 at 02:58, xxxxxxxx wrote:

I wonder if there's something as you say Mohamed that isn't done in the separate render thread? I'm only after a software/viewport like preview image, so nothing fancy. But it's got to be separate from the main thread.

Reading some Python topics on this, it appears others have been able to do it there, unless I've misread them. But haven't seen anything on the c++ side of it. You'd think if Python could do it, c++ could do it as well.

WP.

Helper

On 23/10/2014 at 04:33, xxxxxxxx wrote:

well, it is just a "guess"

test your plugin in a very very complex scene, which takes time to initialize data "like 20 seconds" , scenes that contain lots of geometry and displacement

with this test, "if" the call in separate thread didn't take time to initialize, then my "guess" is correct

Helper

On 23/10/2014 at 11:50, xxxxxxxx wrote:

Hi WikedP,

Did you try the RENDERFLAGS_SHOWERRORS flag? Could you please try it out and report back?

Additionally, I have some rules and limitations about threading in Cinema 4D to tell you. Without all of your code, I don't know which one you might be violating in particular outside of those related to freezing up the application, but I'll still list them all:

- GUI interaction must never happen in a thread. Instead, offload jobs to the main thread queue.

- Input states (e.g. from the keyboard or mouse) must never be queried in a thread.

- Do not write to global structures from a thread.

- Any shared resources must be properly protected by SpinLocks or Semaphores.

- Be aware of how deadlocks are created.

I'll keep digging to try to find what might be wrong, but it's probably rendering without the complete set of data it needs. Otherwise, it's failing to write the data at the end. Without your actual code, I can't determine if it's a subtle error or omission, and same without knowing the results from using the flag mentioned above. Please at least try the flag and report back your findings.

Thanks,

Joey Gaspe
SDK Support Engineer

Helper

On 24/10/2014 at 01:03, xxxxxxxx wrote:

Hi Joey/Mohamed,

Mohamed:

when I render to picture viewer using the same software renderer (and started through the render to picture viewer command), it appears to render quicker than what it does via my custom thread using the same settings. Don't know if that means anything or not.

Joey:

the RENDERFLAGS_SHOWERRORS doesn't seem to report anything that I can see. Are the errors meant to pop up in a MessageDialog or something similar? Or is this what triggers the different RENDERRESULT flags?

Just to address some of the points you've listed:

1: The only gui related code in the thread is a call back to the dialog to redraw the user area (redraw is done via UA->Redraw(TRUE)). This call is only made once during the thread cycle.

2: No input states in code, none necessary to have.

3: Now, what I'm doing is passing a dialog class level MultipassBitmap array reference for the thread to draw too. Does this fit under the "global structure" you mention?

4: Regarding code safety, I have some if() statements in the thread's Main() that are checked before doing any rendering, so that the thread can be stopped if references are null, invalid, don't need drawing etc. All container/object references are built before the dialog is opened, and the thread is only run from the dialog. The dialog is opened via the object plugin. Do any of these containers fall under the point 3 rule too?

5: can't think of any deadlock possibilities, as mentioned, any containers related to the dialog and thread are made before the dialog is opened, and nothing that I can think of needs to wait on the thread to do anything.

Here's the two thread functions that relate to the render process, first one's setting the render settings container which looks like this:

 void MY_THREAD::SetRenderSettings(void)  
 {  
  // RenderSettings & SceneDocument are a thread-class level BaseDocument and BaseContainer
// Time is a thread-class level object
// Resolution_X & Resolution_Y are also thread-class level, and are set prior to the Thread.Start() call  
    
  RenderSettings = SceneDocument->GetActiveRenderData()->GetData();  
  RenderSettings.SetLong(RDATA_RENDERENGINE,RDATA_RENDERENGINE_PREVIEWSOFTWARE);  
  RenderSettings.SetBool(RDATA_SAVEIMAGE,FALSE);  
  RenderSettings.SetReal(RDATA_XRES,Resolution_X);  
  RenderSettings.SetReal(RDATA_YRES,Resolution_Y);  
  RenderSettings.SetBool(RDATA_ALPHACHANNEL,TRUE);  
  RenderSettings.SetBool(RDATA_STRAIGHTALPHA,TRUE);  
  RenderSettings.SetLong(RDATA_FRAMESEQUENCE,RDATA_FRAMESEQUENCE_MANUAL);
  Time.SetDenominator(Frame_Rate);  
  Time.SetNumerator(1);
  RenderSettings.SetTime(RDATA_FRAMEFROM,Time);  
  RenderSettings.SetTime(RDATA_FRAMETO,Time);  
 }

And here's the thread's Main() function:

void MY_THREAD::Main(void)  
 {  
  // SceneDocument is a thread-class level variable  
    
  SceneDocument = GetActiveDocument();
  if(SceneDocument == NULL)  
  {  
   GePrint("SceneDocument is NULL, returning");  
   return;  
  }
  Initial_Go_Frame = Start_Frame;  
  RENDERRESULT Result = RenderDocument(SceneDocument,RenderSettings,NULL,NULL,Frame_Container->operator[](Initial_Go_Frame)->GetLayerNum(1),RENDERFLAGS_EXTERNAL|RENDERFLAGS_SHOWERRORS,this->Get());  // RENDERFLAGS_NODOCUMENTCLONE|
  if(Result == RENDERRESULT_OK)  //  RENDERFLAGS_NODOCUMENTCLONE|RENDERFLAGS_NODOCUMENTCLONE  
  {  
   GePrint("Frame " + LongToString(1) + " complete..");  
  }  
  else  
  {  
   if(Result == RENDERRESULT_USERBREAK)  
   {  
    GePrint("Rendering:User stopped");  
    return;  
   }
   if(Result == RENDERRESULT_OUTOFMEMORY){GePrint("Rendering:Out of memory");}  
   else if(Result == RENDERRESULT_ASSETMISSING){GePrint("Rendering:Asset(s) missing");}  
   else if(Result == RENDERRESULT_SAVINGFAILED){GePrint("Rendering:Failed to save");}  
   else if(Result == RENDERRESULT_GICACHEMISSING){GePrint("Rendering:Missing GI cache");}  
  }
  GePrint("Thread drawing finished..");  
 }

There are some omissions from the above code block, but they are only flag checks and testing for the ok to go ahead and render. One line also missing is the user area redraw as mentioned, but other than that there's nothing else there. I think that's pretty much it regarding the thread. Let me know if you need more to go on.

WP.

Helper

On 24/10/2014 at 02:44, xxxxxxxx wrote:

well, my guess is near confirmation, the thread doesn't has the data "to prepare ray objects, (pre processing stuff)" , so it simply renders an empty scene, which is black, the SDK support may confirm/tell the truth about the behavior here, but it is just a guess because I'm not sure of how it works

in general, the thread must own all data, this is true for main thread so it can render "and give orders to other threads with required data after it prepares RayObjects" , but other threads can't prepare RayObjects because they don't own the data

Helper

On 24/10/2014 at 11:50, xxxxxxxx wrote:

Hi WickedP,

Although I haven't found a specific explanation, I did find more restrictions in terms of threading in Cinema 4D plugins. Lets start with that list:

For all threaded functions it is forbidden to:

- Add an Event.
- Make any changes to materials.
- Change the structure of objects attached to the scene.
- Change parameters of elements attached to the scene (allowed, but not recommended except for tags).
- Call a Draw function.
- During drawing to do any file operations. (During execution it is allowed.)
- Create undos.

Otherwise, there must be a limitation due to thread local storage being used, or some similar issue, such as data on the main thread stack being inaccessible to your render thread. Additionally, I discovered that rendering involves multiple threads outside of your control. Their code is probably written in such a way that the main thread is assumed to be the origin of the RenderDocument() call.

I'm making these assumptions because you state that it works on the main thread without issue. I did a lot of research, but I can't find a more precise explanation right now, but I have a hunch you're stuck rendering from the main thread.

I'll see what else I can dig up, and let me know what this new info might reveal on your side.

Thanks,

Joey Gaspe
SDK Support Engineer

Helper

On 27/10/2014 at 09:16, xxxxxxxx wrote:

Hello Joey,

could you please elaborate on "Add an Event" being forbidden from a threaded context? We've been
using SpecialEventAdd() frequently to notify respective listeners to process data from the main thread.

Thanks
-Niklas

Helper

On 27/10/2014 at 12:46, xxxxxxxx wrote:

Hi NiklasR,

Using SpecialEventAdd() should be fine. It's directly using EventAdd() that would violate the restrictions.

Joey Gaspe
SDK Support Engineer

Helper

On 27/10/2014 at 13:46, xxxxxxxx wrote:

Hi WickedP,

Sebastian gave me some information I'd like to share with you:

- You state that you are drawing a user area from your thread, which is something you shouldn't do.

- You probably shouldn't be calling GetActiveDocument() in a thread. Instead, you should handle the document as an argument that the caller hands over to the thread.

For instance, declare a private member pointer in your thread:
BaseDocument * _doc; // the document to render

Clone the document before starting the thread from a method:
_doc = (BaseDocument * )doc->GetClone(COPYFLAGS_0,nullptr);

Start the thread, set things up for RenderDocument() (target bitmap, etc.), and call it with the following flags:
flags = RENDERFLAGS_NODOCUMENTCLONE|RENDERFLAGS_EXTERNAL

RenderDocument(_doc,renderData,nullptr,nullptr,_bitmap,flags,nullptr,nullptr,nullptr);

After that call, you can add a special event to get around the Draw() restriction, to show the result in the Picture Viewer. It must be called from the main thread.

I limited the amount of code to avoid making the reply too long, but if you need more precise explanations, let me know.

I hope that helps!

Joey Gaspe
SDK Support Engineer

Helper

On 29/10/2014 at 08:26, xxxxxxxx wrote:

Ok, I've had to make some changes to the way I'm running this particular routine that uses the custom thread, but it does appear to be working now. A few small things to iron out still, but I'll do some more digging on those myself before I pester the forum here! Cheers,
WP.

Helper

On 29/10/2014 at 08:42, xxxxxxxx wrote:

Hi WickedP,

I'm glad to hear you've made it work. We're here to help, don't be shy to ask!

Joey Gaspe
SDK Support Engineer