no join() in C4DThread? [SOLVED]

[Helper]

On 25/07/2015 at 05:16, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   15+ 
Platform:   Windows  ;   
Language(s) :     C++  ;

---------
I'm doing a simple image processing, which consumes time, so I put it in a thread "plugin VideoPost"
now C4dThread always crashes or not running correctly

  
							AutoGeFree<SevenPhotonsImageThread> imageThread = NewMemClear(SevenPhotonsImageThread, 1);
							imageThread->kndData = kndData;
							imageThread->input = input;
							imageThread->node_index = n;
							imageThread->shader = shader;
							imageThread->mat = mat;
							imageThread->imageData = imageData;
							imageThread->w = w;
							imageThread->h = h;
  
							imageThreads.push_back(imageThread);
							imageThread->Start();//THREADMODE_SYNCHRONOUS  
                                                        // other stuff, later  
		auto threadIterator = imageThreads.begin();
		while(threadIterator != imageThreads.end())
		{
			if(*threadIterator && (*threadIterator)->IsRunning())
			{
				(*threadIterator)->End();
			}
			imageThreads.erase(threadIterator++);
		}  

the above code fails, and the scenario of using thread pool with a control thread is horrible "I want to launch threads as soon as I get the image, not gathering all images in 1 go then launch"

here is what worked for me "exactly the same code but using C++11 std::thread"

  
							std::thread *sThread = new std::thread(SPImageThread, w, h, shader, mat, imageData);
							stlImageThreads.push_back(sThread);  
                                                        // other stuff, later  
		auto threadIterator = stlImageThreads.begin();
		while(threadIterator != stlImageThreads.end())
		{
			if((*threadIterator)->joinable())
				(*threadIterator)->join();
			delete *threadIterator;
			*threadIterator = nullptr;
  
			stlImageThreads.erase(threadIterator++);
		}  

and this works like a charm, just afraid of exceptions.

[Helper]

On 25/07/2015 at 10:04, xxxxxxxx wrote:

i don't know for sure, but the problem might be storing autogefree objects in a stl container.
autogefree doesn't have an explicit copy constructor or a move constructor so when you push these into a container you now have two objects that both have ownership of the allocated memory (the local autogefree and the copy in the container).
when the local one goes out of scope (i assume you're creating the threads in some kind of loop) it frees the SevenPhotonsImageThread object and at that point all bets are off.

ps: a solution might be to not use autogefree but use std::unique_ptr with a custom deleter that calls deletemem.
pps: i'm not really familiar with all the alloc macros in c4d, but assuming sevenphotonsimagethread is a class extending c4dthread, shouldn't you be using newobj instead?

[Helper]

On 25/07/2015 at 13:16, xxxxxxxx wrote:

Hi droon,

thanks!, you nailed it

here is the new code:

SevenPhotonsImageThread* imageThread = NewObj(SevenPhotonsImageThread); 
    
    
    							imageThread->kndData = kndData;
    
    							imageThread->input = input;
    
    							imageThread->node_index = n;
    
    							imageThread->shader = shader;
    
    							imageThread->mat = mat;
    
    							imageThread->imageData = imageData;
    
    							imageThread->w = w;
    
    							imageThread->h = h;
    
      
    
    
    							imageThreads.push_back(imageThread);
    
    							imageThread->Start();//THREADMODE_SYNCHRONOUS  
                                                            // other stuff, later  
    
    
    		auto threadIterator = imageThreads.begin();
    
    		while(threadIterator != imageThreads.end())
    
    		{
    
    			if(*threadIterator && (*threadIterator)->IsRunning())
    
    			{
    
    				(*threadIterator)->End();
    
    			}
                            DeleteObj(*threadIterator);
    
    			imageThreads.erase(threadIterator++);
    
    		}

it works, but for some reason it glitches Cinema4D a little, so both std::thread and C4DThread are taking 6 seconds to calculate, but C4dThread is hanging GUI "progress bar doesn't update for example, .."

[Helper]

On 25/07/2015 at 23:26, xxxxxxxx wrote:

after some testing, it is not consistent using C4DThread, images sometimes appear black, makes it unusable "as it sometimes generates correct images, sometimes generates some images and others are black, undefined behavior"

[Helper]

On 27/07/2015 at 08:27, xxxxxxxx wrote:

Hi Mohamed,

I think, the strange behavior of your threads will be difficult to diagnose without code.
Also I don't understand, why you don't want to use MPThreadPool. I think, speed-wise it may improve over firing a thread for every image as soon as it is done. But of course that's your decision.

[Helper]

On 27/07/2015 at 10:50, xxxxxxxx wrote:

Hi Andreas,

here is the thread function:

  
void SevenPhotonsImageThread::Main(void)
{
	AutoAlloc<BaseBitmap> bitmap;
	bitmap->Init(w,h, 32);
  
	BaseShader* clone = (BaseShader* )shader->GetClone(COPYFLAGS_0,nullptr);
	RenderShaderPreview(mat->GetDocument()->GetDocumentPath(),clone,shader,this->Get(),bitmap,nullptr,0,RENDER_PREVIEW_USE_BMP_SIZE);//same behavior if I use nullptr in thread argument.
	
	for(Int32 line = 0; line < h; ++line)
		bitmap->GetPixelCnt(0, line, w, &imageData->data[w * line * COLORBYTES_RGB], COLORBYTES_RGB, COLORMODE_RGB, PIXELCNT_0);
	imageData->is_float = false;//because we use uchars, if we use floats then this is true, may be dynamic later
	imageData->width = w;
	imageData->height = h;
	imageData->depth = 1;
	imageData->channels = COLORBYTES_RGB;
  
	BaseShader::Free(clone);
}

sometimes it works, sometimes it gives black images.
the same code in std::thread works without any flaws.

[Helper]

On 28/07/2015 at 07:51, xxxxxxxx wrote:

Hi Mohamed,

still, hard to tell. At a first glance your code looks ok.
I have set up a simple command plugin. On a button press, it creates a BaseArray of C4DThreads (storing active material and its first shader in member variables of the thread), starts all threads and then waits for them to finish. Afterwards it uses ShowBitmap() to display the results, before deleting the thread instances.
The threads do basically the same as yours (without setting all the member variables).
This works just fine here and I did not use any std library functionality.

Can you perhaps try to narrow it down on your side?
Did you check the result of RenderShaderPreview()? Do all threads get correctly started? And did you make sure, your cleanup doesn't kill threads, before they have finished? Assuming imageData is a member of your thread class, does this data get consumed before you destroy the thread instances?

[Helper]

On 28/07/2015 at 09:24, xxxxxxxx wrote:

Hi Andreas,

this is how the imageData is created

  
                                                        images.push_back(SevenPhotonsImageData());  
							SevenPhotonsImageData *imageData = &(images.back());
							Int32 w = kndData->nodes[n].getInternalData(input + 1).GetInt32();
							Int32 h = kndData->nodes[n].getInternalData(input + 2).GetInt32();
							imageData->data.resize(w * h * COLORBYTES_RGB);
  
							const void * address = static_cast<const void*>(imageData);//used inside the renderer later with a function callback to get the image  

images is a std::list<>, it is a member of the VideoPost plugin class.
so what I do is:
allocate memory for imageData inside main thread, take its pointer to my renderer
then fill this memory inside a thread, imageData member variable is just a pointer.
my second reply got some code about how the thread data is filled.

[Helper]

On 28/07/2015 at 09:25, xxxxxxxx wrote:

I didn't check the result of RenderShaderPreview(), will check soon.

[Helper]

On 28/07/2015 at 09:39, xxxxxxxx wrote:

after some testing, threads Start() correctly, RenderShaderPreview() returns RENDERRESULT_OK.
in my test scene, I have 30 textures, checking images with ShowBitmap(bitmap),
last x images (x is any number, 5, 20, 15, ..) got filled randomly "for example 20% scanline fill, 60%, ...".

tested the same code again with std::thread, no errors, all images fills correctly.

[Helper]

On 28/07/2015 at 09:49, xxxxxxxx wrote:

last test was weird to identify the problem:
RenderShaderPreview(mat->GetDocument()->GetDocumentPath(),clone,shader,this->Get(),bitmap,nullptr,0,RENDER_PREVIEW_USE_BMP_SIZE) //undefined behavior, random fill.

RenderShaderPreview(mat->GetDocument()->GetDocumentPath(),clone,shader,nullptr,bitmap,nullptr,0,RENDER_PREVIEW_USE_BMP_SIZE) //works fine

[Helper]

On 28/07/2015 at 10:01, xxxxxxxx wrote:

Internally the thread pointer is used to interrupt/break the thread.
I think, this is might be a hint, that your threads get aborted too early for some reason.

One more thing, you might already know: ShowBitmap() needs to called from the main thread.

[Helper]

On 28/07/2015 at 10:41, xxxxxxxx wrote:

aha, I see now the problem!!
when I have too many images, each thread got queue of tasks "function calls" , which are executed in a loop.
when I call (*threadIterator)->End(), it exits without finishing the current function, I see that happens from the pointer as you mentioned, via signal of thread break.

I changed (*threadIterator)->End() to (*threadIterator)->Wait() and it works fine now
thanks Andreas for the help , you can consider this solved.