Texture baking crashes

Helper

On 14/10/2016 at 07:38, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   R18
Platform:   Windows ;
Language(s) :     C++ ;

---------

Hello.

I have a MaterialData plugin which uses texture baking in procedural shaders when they are sent to my rendering engine.
Even though it works fine with noise, gradient, fresnel, e.t.c. , I get random crashes when a shader has it's own shader parameter internally (e.g. Filter shader).
The problem is that it's not always reproducable. I also feel like that way I do things in texture baking is very wrong.

Here is what I do in pseudocode.
1. create a temp document BaseDocument (via Alloc)
2. create a primitive plane inside the document (via GeneratePrimitive)
3. use SendModelingCommand(MCOMMAND_CURRENTSTATETOOBJECT, ... ) on the plane.
4. create a texture tag (Alloc) and add it to the plane.
5. create a C4D material
6. Copy the shader that we want to bake in Color channel of the C4D material.
7. call InitBakeTexture(temp_document, texture_tag, uvw_tag, ...) //The ones we created above.
8. create a multipass bitmap that will store the baked texture
9. call BakeTexture(...)
10. crash

Here is the actual code:

BaseList2D* getParameterLink(GeListNode &node, LONG paramID, LONG instanceOf){
    GeData parameter;
    if (node.GetParameter(DescLevel(paramID), parameter, DESCFLAGS_GET_0)) {
        BaseLink* link = parameter.GetBaseLink();
        if (link)
            return link->GetLink(node.GetDocument(), instanceOf);
    }
    return 0;
}

//step 1
BaseDocument* temp_document = BaseDocument::Alloc();
  
//step 2
BaseContainer plane_data;
plane_data.InsData(PRIM_PLANE_SUBW,GeData(1));
plane_data.InsData(PRIM_PLANE_SUBH,GeData(1));
BaseObject* plane = GeneratePrimitive(temp_document, Oplane, plane_data, 1.0, false, thisThread);
ModelingCommandData command_data;
command_data.doc = temp_document;
command_data.op = plane;
  
//step 3
if (!SendModelingCommand(MCOMMAND_CURRENTSTATETOOBJECT,command_data)) {
	BaseObject::Free(plane);
	return false;
}
BaseObject::Free(plane);
BaseObject* polygon_plane = static_cast<BaseObject *>(command_data.result->GetIndex(0));
bake_uvw_tag = (UVWTag * )polygon_plane->GetTag(Tuvw,0);
  
//step 4
bake_texture_tag = TextureTag::Alloc();
bake_texture_tag->SetParameter(DescID(TEXTURETAG_PROJECTION),TEXTURETAG_PROJECTION_UVW,DESCFLAGS_SET_0);
polygon_plane->InsertTag(bake_texture_tag);   
temp_document->InsertObject(polygon_plane,NULL,NULL);
  
//step 5
Material* bake_material = Material::Alloc();
for (LONG j = CHANNEL_COLOR; j<CHANNEL_NORMAL; j++) { //disable all channels
	bake_material->SetChannelState(j,FALSE);
}
bake_material->SetChannelState(CHANNEL_COLOR,TRUE); //Color only
temp_document->InsertMaterial(bake_material); 
  
//step 6
LONG type = 0;
BaseList2D* shaderLink = getParameterLink(*original_material, shader_for_baking);  
if (shaderLink) {
	BaseShader* cloned_shader = (BaseShader * )shaderLink->GetClone(COPYFLAGS_0,NULL);
	type = cloned_shader->GetType();
	BaseContainer *data = bake_material->GetDataInstance();
	data->SetLink(MATERIAL_COLOR_SHADER, cloned_shader);
	bake_material->InsertShader(cloned_shader, NULL);
	bake_material->Message(MSG_UPDATE);
	bake_material->Update(true, true);
}
bake_texture_tag->SetMaterial(bake_material); 
  
BaseContainer bake_settings;
bake_settings.InsData(BAKE_TEX_WIDTH,GeData(400));
bake_settings.InsData(BAKE_TEX_HEIGHT,GeData(400));
bake_settings.InsData(BAKE_TEX_UV_LEFT,GeData(0.0));
bake_settings.InsData(BAKE_TEX_UV_RIGHT,GeData(1.0));
bake_settings.InsData(BAKE_TEX_UV_TOP,GeData(0.0));
bake_settings.InsData(BAKE_TEX_UV_BOTTOM,GeData(1.0));
bake_settings.InsData(BAKE_TEX_AMBIENT_OCCLUSION,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_ILLUMINATION,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_SHADOWS,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_NO_GI,GeData(TRUE));
bake_settings.InsData(BAKE_TEX_USE_CAMERA_VECTOR,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_SHOW_STATUS,GeData(TRUE));
bake_settings.InsData(BAKE_TEX_PROGRESS_BITMAP,GeData(TRUE));
bake_settings.InsData(BAKE_TEX_COLOR,GeData(TRUE)); //Color only
bake_settings.InsData(BAKE_TEX_DIFFUSION,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_LUMINANCE,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_ALPHA,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_TRANSPARENCY,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_REFLECTION,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_DISPLACEMENT,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_NORMAL,GeData(FALSE));
bake_settings.InsData(BAKE_TEX_BUMP,GeData(FALSE));
  
//step 7
BAKE_TEX_ERR bake_result = BAKE_TEX_ERR_NONE;
BaseDocument* bake_document = InitBakeTexture(temp_document, bake_texture_tag, bake_uvw_tag, NULL, bake_settings, &bake_result, NULL);
  
//step 8
MultipassBitmap * bake_bitmap = MultipassBitmap::Alloc(400, 400, COLORMODE_RGB); 
if (bake_bitmap) {
	//step 9
	bake_result = ::BakeTexture(bake_document, bake_settings, bake_bitmap, NULL, NULL, NULL); //crash
	.....
}	
//Free Multipass
//Free bake document
//Free texture tag

The reason why I create the initial plane is to produce the default UVW coordinates and always bake the texture based on these. Sadly, I have to do this procedure for EACH MATERIAL, which seems very very slow. is this the correct way to bake textures in my MaterialData shader link parameters ?
Which steps can I discard to make it faster ?

If you need any more information, please let me know so I can provide them asap.

Thank you very much for your time.

Helper

On 17/10/2016 at 04:05, xxxxxxxx wrote:

Hello.

Here are some more information.

The InitBakeTexture returns with no error
BakeTexture never returns.
Here is the stack on pausing BakeTexture:
[External Code]
c4dplugin.xdl64!00007ff8fb2754b4() Unknown
c4dplugin.xdl64!00007ff8f9371915() Unknown
c4dplugin.xdl64!00007ff8f923f71e() Unknown
c4dplugin.xdl64!00007ff8f947ec87() Unknown
c4dplugin.xdl64!00007ff8f94567af() Unknown
c4dplugin.xdl64!00007ff8fb0cd0e3() Unknown
c4dplugin.xdl64!00007ff8f9483553() Unknown
sla.cdl64!00007ff8e8ec26e9() Unknown
sla.cdl64!00007ff8e8ec0e39() Unknown
it occurs even when I set
bake_settings.InsData(BAKE_TEX_COLOR,GeData(FALSE));
it mostly occurs when I switch documents and it is caused in InitGLImage of my MaterialData plugin.

Thank you.

Helper

On 17/10/2016 at 09:36, xxxxxxxx wrote:

Hi,

using your code I have not been able to reproduce the issue here, yet.
First question, where (i.e. in which function) do you call that code?
Maybe you could also provide me with a simple scene, so I can test on the same data. Simply send one to [email protected]

There are a few things in your code. Maybe it's just stripped down demo code, but I should mention anyway:
- You allocate a temp_document in the beginning. In the SendModelingCommand() error case it's not free'd, causing a memory leak in this case. I recommend to use AutoAlloc() as much as possible to avoid such issues.
- There are a lot of nullptr checks missing.
- Finally, a possible reason for your crashes: In the end you have a comment saying "Free texture tag". But you inserted the bake_texture_tag on the plane object. From that point you are no longer owner of this tag. If you'd free'd that tag in the place of the comment in the end, a crash is inevitable sooner or later.

Helper

On 17/10/2016 at 10:24, xxxxxxxx wrote:

Hello and thank you very much for your help.

The code is indeed stripped down removing all the checks.
Thank you very much for your comment in Free Texture Tag...

Also, in the part where I clone the shader, if that shader is a filter which contains another shader, is that cloned and inserted in the document as well ? What happens in this case ?

Thank you again.

Helper

On 18/10/2016 at 09:33, xxxxxxxx wrote:

Yes, sub-shaders are cloned as well, the parent shader owns them. You shouldn't need to worry about these.

Helper

On 19/10/2016 at 05:11, xxxxxxxx wrote:

Hello.

I have solved all the crashes thanks to you help.

The only issue that remains is related to memory leak during texture baking.
If I add the following code in a while(1) loop, the memory keeps increasing.
So I assume I don't handle the deallocation correctly. Do I miss anything after BakeTexture call ?

  
BaseDocument* bake_document = InitBakeTexture(temp_document, bake_texture_tag, bake_uvw_tag, NULL, bake_settings, &bake_result, NULL);
if (bake_document) {
    if (bake_result == BAKE_TEX_ERR_NONE) {
        MultipassBitmap* bake_bitmap = MultipassBitmap::Alloc(resolution_x, resolution_y, COLORMODE_RGB); 
        if (bake_bitmap) {
            bake_result = BakeTexture(bake_document, bake_settings, bake_bitmap, NULL, NULL, NULL);
            MultipassBitmap::Free(bake_bitmap);
        }
    }
    BaseDocument::Free(bake_document);
}

Thank you.