InitRender() with large textures

rsodre

Hi,

There was an open issue on the old forums that was closed with it.
https://developers.maxon.net/forum/topic/10904

The problem was a InitRender() failure with textures larger than 2048x2048.
I still need some direction on what to do with it.

Thanks.

a_block

Hi,

I'm terribly sorry, this topic was left open in the old forum.
As I said there, I can reproduce the issue. It is under investigation in development and I hope we'll get a fix in future. Unfortunately I don't have a workaround to offer currently.

I have turned your thread into a question, see Q&A New Functionality.

Cheers,
Andreas

rsodre

@a_block Hi Andreas,

Any news on this issue?

roger

a_block

Hi,

unfortunately not. When I saw your message, I contacted development again, and it is under investigation. But I was not given any estimates. But your post is still on my list of unresolved ones and I haven't forgotten about it. I'll try to keep you updated as soon as there are any news.

Cheers,
Andreas

rsodre

This problem still persists, and sometimes with smaller textures (1024x1024).

The shader is declared from a resource file (SHADERLINK), and I need to initialize and sample it in a modifier's ModifyObject(). It's reproductible always when using a 4096x4096 texture, when loading the project. After the project is fully loaded, I can initialize and use the shader, but the first time always fail.

I can see some new information on the XCode console after calling InitRender():
Application transferred too few scanlines

Does it give any clues?

Paul Everett

make sure the bitmap shader is not set to MIP mapping, then it works fine.

rsodre

@Paul-Everett I had strand shader samples before that were fixed by changing the Sampling, but for that particular issue, if i save the project with a 4K texture and Sampling other than MIP, when the project loads the error also happens.

r_gigante

Hi @rsodre, I'm sorry for being this issue still pending on your side and I apologize for having forgot this opened topic.

I've tried to catch up with the discussion you had with Andreas and I need some help of yours since I can't reproduce the issue in my test plugin under any circumstance.

What i tried so far was:

use different textures (from 2048x2048 to 10240x10240) in different formats (.jpg / .png) and different color depth
call BaseShader::InitRender() and BaseShader::Sample() either in a ObjectData::ModifyObject() method and/or in a TagData::Execute()
save and re-open the scene containing the test ObjectData and TagData are used
run the same code either on R19 and R20
run the same code either on Windows and macOS

Can you help me reproduce the issue?

Best, Riccardo

rsodre

Hi @r_gigante

Yes, those are the steps to reproduce.
The problem happens when the file loads, InitRender() will fail with INITRENDERRESULT_USERBREAK.

r_gigante

Hi Roger, can you please provide a texture to test here?

Thanks, Riccardo

rsodre

@r_gigante DistributionMap_4K.jpg

r_gigante

Hi Roger, thanks for following up and providing the texture.

A few notes:

the texture attached actually points to a resized texture (760 x 760): is it wanted?
the texture gets loaded and sampled flawlessly on my test plugins (both ObjectData and TagData)
test were done on macOS/C4DR19.024

Could you please provide the full high-res picture to grant correct testing?

Cheers, Riccardo

rsodre

@r_gigante I uploaded the texture here on the forums, I think it was reduced. Here's it is.
But any JPG, 4096x4096, using MIP sampling fails with me.

r_gigante

Hi Roger, thanks for providing the test picture.

Unfortunately I've not being able to reproduce the issue with the test code below.
Could you kindly check the same code on your side and see how it behaves?

ObjectData::Modify()

Bool PC_11085_ObjectData::ModifyObject(BaseObject* mod, BaseDocument* doc, BaseObject* op, const Matrix& op_mg, const Matrix& mod_mg, Float lod, Int32 flags, BaseThread* thread)
{
	if (!mod || !op || !doc || !thread)
		return false;
	
	// Retrieve the BaseContainer instance of the modifier.
	BaseContainer *bcPtr = mod->GetDataInstance();
	
//	// Retrieve the Object Manager parameters values.
	const Bool useXbitmap = bcPtr->GetBool(PC11085_XBITMAP);
	BaseList2D* const xBitmapBL = bcPtr->GetLink(PC11085_SHADERLINK, doc);
	const String textureFilename = bcPtr->GetString(PC11085_TEXTURE);
	
	// allocate and init the InitRenderStruct
	InitRenderStruct irs(doc);
	
	// check the color space
	COLORSPACETRANSFORMATION transform = COLORSPACETRANSFORMATION_NONE;
	// check if linear workflow is enabled
	if (irs.linear_workflow)
		transform = COLORSPACETRANSFORMATION_LINEAR_TO_SRGB;
	
	if (xBitmapBL)
	{
		// cast to BaseShader
		BaseShader* activeMatColorBS = (BaseShader*)xBitmapBL;
		
		// define the ChannelData and set the u/v coords for sampling
		ChannelData chData;
		
		// call the InitRender before executing sampling
		const INITRENDERRESULT res = activeMatColorBS->InitRender(irs);
		if (res != INITRENDERRESULT_OK)
			return EXECUTIONRESULT_OK;
		
		const Int32 steps = 100;
		
		for (Int32 x = 0; x < steps; x++)
		{
			for (Int32 y = 0; y < steps; y++)
			{
				chData.p.x = double(x)/double(steps);
				chData.p.y = double(y)/double(steps);
				// sample
				const Vector sampledValue = activeMatColorBS->Sample(&chData);
				// correct based on the color space transformation
				const Vector transformedColor = TransformColor(sampledValue, transform).Clamp01();
				// just print
				GePrint("Shader sample at ["+String::FloatToString(chData.p.x)+","+String::FloatToString(chData.p.y)+"]: "+String::VectorToString(sampledValue)+" / "+String::VectorToString(transformedColor));
			}
		}
		
		
		// call the FreeRender to release allocated memory used for sampling
		activeMatColorBS->FreeRender();
	}
	
	// Notify Cinema about the internal data update.
	op->Message(MSG_UPDATE);
	
	return true;
}

TagData::Execute()

EXECUTIONRESULT PC_11085_TagData::Execute (BaseTag *tag, BaseDocument *doc, BaseObject *op, BaseThread *bt, Int32 priority, EXECUTIONFLAGS flags)
{
	if (!tag || !doc || !op || !bt)
		return EXECUTIONRESULT_OUTOFMEMORY;
	
	// allocate and init the InitRenderStruct
	InitRenderStruct irs(doc);
	
	// check the color space
	COLORSPACETRANSFORMATION transform = COLORSPACETRANSFORMATION_NONE;
	// check if linear workflow is enabled
	if (irs.linear_workflow)
		transform = COLORSPACETRANSFORMATION_LINEAR_TO_SRGB;
	
	// find for the attached TextureTag and the related material
	TextureTag* opTTag = (TextureTag*)op->GetTag(Ttexture);
	if (!opTTag)
		return EXECUTIONRESULT_OK;
	
	BaseMaterial* opMat = opTTag->GetMaterial();
	if (!opMat)
		return EXECUTIONRESULT_OK;
	
	// look for the material BaseContainer and the BaseList2D associated to the color param
	BaseContainer* activeMatBC = opMat->GetDataInstance();
	if (!activeMatBC)
		return EXECUTIONRESULT_OK;
	
	BaseList2D* activeMatColorBL = activeMatBC->GetLink(MATERIAL_COLOR_SHADER, doc);
	if (!activeMatColorBL)
		return EXECUTIONRESULT_OK;
	
	// cast to BaseShader
	BaseShader* activeMatColorBS = (BaseShader*)activeMatColorBL;
	
	// define the ChannelData and set the u/v coords for sampling
	ChannelData chData;
	
	// call the InitRender before executing sampling
	const INITRENDERRESULT res = activeMatColorBS->InitRender(irs);
	if (res != INITRENDERRESULT_OK)
		return EXECUTIONRESULT_OK;
	
	const Int32 steps = 100;
	
	for (Int32 x = 0; x < steps; x++)
	{
		for (Int32 y = 0; y < steps; y++)
		{
			chData.p.x = double(x)/double(steps);
			chData.p.y = double(y)/double(steps);
			// sample
			const Vector sampledValue = activeMatColorBS->Sample(&chData);
			// correct based on the color space transformation
			const Vector transformedColor = TransformColor(sampledValue, transform).Clamp01();
			// just print
			GePrint("Shader "+activeMatColorBL->GetName()+" sample at ["+String::FloatToString(chData.p.x)+","+String::FloatToString(chData.p.y)+"]: "+String::VectorToString(sampledValue)+" / "+String::VectorToString(transformedColor));
		}
	}
	
	
	
	// call the FreeRender to release allocated memory used for sampling
	activeMatColorBS->FreeRender();
	
	return EXECUTIONRESULT_OK;
}

Looking forward hearing from you, give best
Riccardo

r_gigante

Hi @rsodre, any feedback so far?

Cheers, Riccardo

rsodre

Hi @r_gigante, I still didn't got a chance to test your sample code and see if it fails here, but it looks like the same workflow I'm using.

When I first opened the issue, I was using a tag, that now evolved to be a modifier, so there's no need to test the tag.
But is your object a modifier?

r_gigante

@rsodre as said in InitRender() with large textures:

Bool PC_11085_ObjectData::ModifyObject

which indeed is only available in BaseObject for modifier.

Look forward your feedbacks

r_gigante

Hi @rsodre, I'm going to mark this thread as solved being passed two weeks from the last post.

Feel free to open a new thread if issue persists.

Best, Riccardo

rsodre

@r_gigante That's sad, because it's a recurrent issue we have.
I'll try to isolate it as soon as I have some time.

rsodre

Hi @r_gigante,

I finally took some time to isolate my shader init and sample code on a simple project, and was unable to reproduce the error I was having. If it works on a separate project, should work on my full plugin too.
So I started stripping code from my modifier and generator, always testing if the shader was loading properly, until the objects were barely empty, when the error finally stopped. I traced the culprit down to a single EventAdd() call in some function inside my generator's ModifyObject(). Removing it fixed the problem. Looks like calling it before the first modifier pass somehow causes the shader init error.
Just wanted to leave this feedback in case anyone has a similar problem someday.
Thanks for the help!