Overlapping images with transparency with BaseBitmap
-
In case you are still stuck, drop me a note here
@ferdinand Just a little update to inform you that, after spending half of my week-end trying to simply draw 1 semi-transparent image (as illustrated in my original message) onto the gray background using the Image API, that I am very much stuck
I'm not sure what you will say in regards to my last message, because if, in the end, the Image API does not allow me to rotate/scale/smooth out the image, then attempting to implement it (i.e. what I've been doing this week-end) might end up being a fruitless endeavor.
Thanks again!
-
Hey @sasha_janvier,
I today spent some time with this but I have not yet an answer yet, as I myself ran into some troubles with this, both from the classic and maxon API side. May I ask on which version of Cinema 4D you are? I assume 2024.2.0?
Regarding your questions:
- Scaling images is possible in the classic Image API through
BaseBitmap::ScaleIt
. The maxon Image API also has image scaling methods, but they are unfortunately not public. - There are no image rotation methods in the classic or maxon Image API, both in their private and public parts.
There is something fishy going on with alphas in 2024.2, at least I can neither get the PNG nor the TIF image loader to do what I think they should do, no matter if I use the classic or maxon API. I will have to do more testing in the next days. I will answer here latest until Friday the 2nd.
Cheers,
Ferdinand - Scaling images is possible in the classic Image API through
-
Thank you very much @ferdinand. It's both relieving and admittedly a bit annoying to have confirmation that something fishy is going on with alphas in 2024.2 given the time I've put into this so far, but ultimately, I'm just glad I brought this issue to you and the team's attention.
And yes, I am using 2024.2.
Regarding your answers, while I knew of the "BaseBitmap::ScaleIt", I guess what I was really asking was whether the Image API still would allow me to do pixel-per-pixel transformations such as scaling/rotating (like I am currently doing with the BaseBitmap class), but upon closer inspection, I feel like the answer to this question would lean towards "Yes".
Thanks again @ferdinand and very much looking forward to your testing and update. I really appreciate it your time and assistance.
-
@sasha_janvier said in Overlapping images with transparency with BaseBitmap:
I guess what I was really asking was whether the Image API still would allow me to do pixel-per-pixel transformations such as scaling/rotating.
Yes that would be possible, at least in one direction. You can create a
BaseBitmap
, then get its underlyingImageRef
, manipulate the image data with the Image API to you hearts content, and then at any point return to the classic APIBaseBitmap
interface and use its methods.BaseBitmap::GetImageRef
returns a reference to the internalImageInterface
used by theBaseBitmap
to store its data.The Cinema 4D API has two worlds as you surely have noticed by now, the so called classic API, that is the code that has been written since version R6-ish of Cinema 4D and mostly uses old-school C++ concepts, and then there is the so called maxon API, that is code that started out around R20 (there is no clear starting point) which uses a more modern approach to C++ (templating, yay!).
Because the current code base of Cinema 4D is about 25 years old, we often cannot replace all code of something in the classic API which we want to replace with something in the maxon API. We then rewrite the original type as an adapter (a 'wrapper') which under the hood uses the new maxon API type. Examples for this are images (
BaseBitmap
,ImageInterface
), URLs/paths (Filename
,UrlInterface
), or color profiles (ColorProfile
,ColorProfileInterface
).In some cases it is then possible to get the underlying maxon API data for a classic API wrapper, e.g., get the
ImageRef
for aBaseBitmap
and with that use both worlds in tandem. However, it usually is not possible to go the other way around, construct a classic API type instance around an existing maxon API type.So, when you start out by loading an image with
BaseBitmap::InitWith
, you can use both worlds. But if you start out withmaxon::ImageTextureInterface::LoadTexture
you cannot.Cheers,
Ferdinand -
FYI: I wrote a mail to the developers who own the image stuff, it might take some time before it has taken its route.
-
Thank you very very much once again, @ferdinand. I really appreciate the thorough explanation, as well as you sending an email to the team at Maxon.
In the meantime, I'll keep using the
BaseBitmap
interface and use an image with no alpha. Not an ideal situation, but at least I can progress with my plugin in the meantime.Thanks a million once again. Very much looking forward to see what Maxon says and how we can resolve this issue.
-
Hello @sasha_janvier,
please excuse the longer waiting time. I have now a solution, but it is unfortunately not a pretty one. What I implied above, simply rely on the Image API and either use
MultipassBitmap
orImageTextureInterface
to load and layer images does not work in the context of images with embedded alpha information. What also does not work is using high level methods such asBaseBitmap::Get/SetPixelCnt
to read and write data, as just for pure loading and copying, you cannot simply treat things asCOLORMODE::ARGB
in this case. Find my litany of failed attempts at the very end. One of the developers of the Image API had a go at this too and could not make it work either usingMultipassBitmap
and pure loading.Solution
What you did in your initial posting was quite close to what you have to do, and is the only thing that works for me. To not make this thread enormous, I have ignored your sub-questions of scaling and rotating images. Please open a new thread for them if you still need help there.
What you could do in addition to my code below, is combine it with
BaseBitmap::Get/SetPixelCnt
, but then deal with the color and alpha bitmap on its own, you cannot just try to flat-out copy/write things asARGB
. You could also useMultipassBitmap
by writing data simply into layers so that you do not have to do the blending on your own. Regarding the predefined blending functions you asked for, there ismaxon::BlendColor
but that just linearly interpolates two colors. There is gfx_image_blend_functions.h which more what you need but I did not end up using it.When loading images with embeded alphas in the Picture Viewer of Cinema 4D, we must enable Image Transparency which is by default off for some reason.
Cheers,
FerdinandInputs
Output
Code
#include "apibasemath.h" #include "c4d_basebitmap.h" #include "maxon/url.h" maxon::Result<void> BlendBitmap() { // This is the exit point for errors in this function, e.g., iferr_return or return maxon::...Error. // It only works because the return type of this method is of the scheme Result<T>, i.e., it returns // an error or an instance of T, here void. iferr_scope; // If we want to use error handling but do not want this function to be of type Result<void> but // for example bool, we would do this: //iferr_scope_handler{ // // Not necessary but nicer, dumps the error prefixed by the function name to the console. // DiagnosticOutput("@: @", MAXON_FUNCTIONNAME, err); // return false; //}; // Define the image size and two input and one output file next to this source file. const Int32 size = 256; const maxon::Url directory = maxon::Url(maxon::String(MAXON_FILE)).GetDirectory(); const maxon::Url urlTex0 = (directory + "bar0.png"_s) iferr_return; const maxon::Url urlTex1 = (directory + "bar1.png"_s) iferr_return; const maxon::Url urlResult = (directory + "out_blend.png"_s) iferr_return; // Auto allocate the bitmap into which we will layer things and add an alpha channel. Auto // allocation is scope based, i.e., #composition will be destroyed when this function is exited. // When we wanted to pass #composition to the outside and give ownership to the caller, we would // use BaseBittmap::Alloc instead. AutoAlloc<BaseBitmap> composition; if (composition->Init(size, size, 24) != IMAGERESULT::OK) return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not load file into bitmap."_s); BaseBitmap* const compChannel = composition->AddChannel(true, false); // Defines a function to layer #filePath into #composition. auto AddLayer = [&compChannel](BaseBitmap* const composition, const maxon::Url& filePath) -> maxon::Result<void> { // Error scope for this lambda and create the bitmap to load #filePath into. iferr_scope; AutoAlloc<BaseBitmap> source; if (source->Init(MaxonConvert(filePath)) != IMAGERESULT::OK) return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not load file into bitmap."_s); // Figure out the the smaller of the two images #composition and #source. const Int32 copyWidth = Min(composition->GetBw(), source->GetBw()); const Int32 copyHeight = Min(composition->GetBh(), source->GetBh()); if (copyWidth < 1 || copyHeight < 1) return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION, "Composition or source has null width or height."_s); // Make sure the source has an alpha channel. BaseBitmap* sourceChannel = source->GetInternalChannel(); if (!sourceChannel) sourceChannel = source->AddChannel(true, false); // Two variables to hold pixel alpha values from the comp and the source and a constant to // convert pixel values from the [0, 255] to the [0, 1] interval. UInt16 a0, a1; const Float32 floatConversion = 255.99f; // Here comes the slow part, we really have to iterate pixel by pixel so that we can splice // channels. for (Int32 y = 0; y < copyHeight; y++) { for (Int32 x = 0; x < copyHeight; x++) { // Get the color and alpha component for the current pixel of both #comp and #source const Vector32 compColor = composition->GetPixelDirect(x, y); const Vector32 sourceColor = source->GetPixelDirect(x, y); composition->GetAlphaPixel(compChannel, x, y, &a0); source->GetAlphaPixel(sourceChannel, x, y, &a1); // Convert the alpha values to the [0, 1] interval. const Float32 compAlpha = Float32(a0) / floatConversion; const Float32 sourceAlpha = Float32(a1) / floatConversion; // Compute the sum of the alphas, i.e., the current alpha value at (x, y) when both images // are layered. And compute a scaling factor for normalized alpha values, i.e., alpha values // where the BG and FG always sum to 1. const Float32 alphaSum = Clamp01(compAlpha + sourceAlpha); const Float32 alphaFactor = 1.f / alphaSum; // Compute the final color and the final alpha value. I am not a big expert on all the bitmap // stuff, but this seems to be the most sensible way to do this. Vector32 color = (compColor * compAlpha * alphaFactor + sourceColor * sourceAlpha * alphaFactor); Int32 alphaInt = ClampValue(Int32(alphaSum * floatConversion), 0, 255); // Comment out to see what is happening, will of course make things very slow. // ApplicationOutput("x:@, y:@, a0:@, a1:@, alpha:@ (@)", x, y, a0, a1, alphaSum, alphaInt); // Write the pixel in the comp. composition->SetPixel(x, y, color.x, color.y, color.z); composition->SetAlphaPixel(compChannel, x, y, alphaInt); } } return maxon::OK; }; // Add a layer for urlTex0 and then one for urlTex1. AddLayer(composition, urlTex0) iferr_return; AddLayer(composition, urlTex1) iferr_return; // Display the image in the Picture Viewer and save it as a PNG. ShowBitmap(composition); composition->Save(MaxonConvert(urlResult), FILTER_PNG, nullptr, SAVEBIT::ALPHA); // In the maxon API, Result<void> functions return maxon::OK when everything went well. return maxon::OK; }
Graveyard
I have put this here so that future readers can see what as of 2024.2 and its following release DOES NOT WORK. All this code is non-functional regarding loading embedded alphas:
#include "apibasemath.h" #include "c4d_basebitmap.h" #include "maxon/gfx_image.h" #include "maxon/url.h" #include "maxon/mediasession_image_export_psd.h" maxon::Result<void> ConstructMultipassBitmap1() { // Does not work in the alpha aspect of lading images. iferr_scope; const Int32 size = 256; const maxon::Url directory = maxon::Url(maxon::String(MAXON_FILE)).GetDirectory(); const maxon::Url urlTex0 = (directory + "bar0.png"_s) iferr_return; const maxon::Url urlTex1 = (directory + "bar1.png"_s) iferr_return; const maxon::Url urlResult = (directory + "out_multi1.psd"_s) iferr_return; MultipassBitmap* composition = MultipassBitmap::Alloc(size, size, COLORMODE::ARGB); if (!composition) return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION); finally { MultipassBitmap::Free(composition); }; for (maxon::Url path : {urlTex0, urlTex1}) { AutoAlloc<BaseBitmap> source; if (source->Init(MaxonConvert(path)) != IMAGERESULT::OK) return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION); MultipassBitmap* const layer = composition->AddLayer(nullptr, COLORMODE::ARGB); MultipassBitmap::AllocWrapper(source)->CopyTo(layer); //const BaseBitmap* const sourceAlpha = source->GetInternalChannel(); //BaseBitmap* layerAlpha = layer->GetInternalChannel(); //if (!layerAlpha) // layerAlpha = layer->AddChannel(true, false); //if (sourceAlpha && layerAlpha) // sourceAlpha->CopyTo(layerAlpha); //else // ApplicationOutput("@: Could not copy alpha channel.", MAXON_FUNCTIONNAME); layer->SetParameter(MPBTYPE::SHOW, true); layer->SetParameter(MPBTYPE::SAVE, true); } ShowBitmap(composition); composition->Save(MaxonConvert(urlResult), FILTER_PSD, nullptr, SAVEBIT::MULTILAYER | SAVEBIT::ALPHA); return maxon::OK; } maxon::Result<void> ConstructMultipassBitmap2() { // Does not work in the alpha aspect of lading images. iferr_scope; const Int32 size = 256; const maxon::Url directory = maxon::Url(maxon::String(MAXON_FILE)).GetDirectory(); const maxon::Url urlTex0 = (directory + "bar0.png"_s) iferr_return; const maxon::Url urlTex1 = (directory + "bar1.png"_s) iferr_return; const maxon::Url urlResult = (directory + "out_multi2.psd"_s) iferr_return; MultipassBitmap* composition = MultipassBitmap::Alloc(size, size, COLORMODE::ARGB); if (!composition) return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION); finally { MultipassBitmap::Free(composition); }; auto AddLayer = []( MultipassBitmap* const composition, const maxon::Url& filePath) -> maxon::Result<MultipassBitmap* const> { iferr_scope; if (!composition) return maxon::NullptrError(MAXON_SOURCE_LOCATION, "Invalid composition pointer."_s); AutoAlloc<BaseBitmap> source; if (source->Init(MaxonConvert(filePath)) != IMAGERESULT::OK) return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not load file into bitmap."_s); MultipassBitmap* const layer = composition->AddLayer(nullptr, COLORMODE::ARGB); const Int32 copyWidth = Min(source->GetBw(), layer->GetBw()); const Int32 copyHeight = Min(source->GetBh(), layer->GetBh()); if (copyWidth < 1 || copyHeight < 1) return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION, "Composition or source has null width or height."_s); iferr (UChar * buffer = NewMem(UChar, (COLORBYTES_ARGB * copyWidth))) return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not allocate copy buffer"_s); BaseBitmap* const sourceAlpha = source->GetInternalChannel(); sourceAlpha->Save(Filename("e:\\alpha.psd"), FILTER_PSD, nullptr, SAVEBIT::NONE); for (Int32 y = 0; y < copyHeight; y++) { source->GetPixelCnt(0, y, copyWidth, buffer, COLORBYTES_ARGB, COLORMODE::ARGB, PIXELCNT::NONE); layer->SetPixelCnt(0, y, copyWidth, buffer, COLORBYTES_ARGB, COLORMODE::ARGB, PIXELCNT::NONE); } DeleteMem(buffer); return layer; }; AddLayer(composition, urlTex0) iferr_return; AddLayer(composition, urlTex1) iferr_return; ShowBitmap(composition); composition->Save(MaxonConvert(urlResult), FILTER_PSD, nullptr, SAVEBIT::MULTILAYER | SAVEBIT::ALPHA); return maxon::OK; } maxon::Result<void> ConstructImageTexture() { // Does not work in the alpha aspect of lading images. iferr_scope; const maxon::Url directory = maxon::Url(maxon::String(MAXON_FILE)).GetDirectory(); const maxon::Url urlTex0 = (directory + "bar0.png"_s) iferr_return; const maxon::Url urlTex1 = (directory + "bar1.png"_s) iferr_return; const maxon::Url urlResult = (directory + "out_imgtexture.psd"_s) iferr_return; const maxon::ImageTextureRef tex0 = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return; const maxon::ImageTextureRef tex1 = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return; tex0.Load(urlTex0, maxon::TimeValue(), maxon::MEDIASESSIONFLAGS::NONE) iferr_return; tex1.Load(urlTex1, maxon::TimeValue(), maxon::MEDIASESSIONFLAGS::NONE) iferr_return; maxon::ImageBaseRef tex0img = tex0.GetFirstChild(maxon::ConstDataPtr(maxon::IMAGEHIERARCHY::IMAGE)); maxon::ImageBaseRef tex1img = tex1.GetFirstChild(maxon::ConstDataPtr(maxon::IMAGEHIERARCHY::IMAGE)); tex0img.Remove(); tex1img.Remove(); const maxon::ImageTextureRef layers = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return; layers.AddChildren(maxon::IMAGEHIERARCHY::IMAGE, tex0img, maxon::ImageBaseRef()) iferr_return; layers.AddChildren(maxon::IMAGEHIERARCHY::IMAGE, tex1img, tex0img) iferr_return; const maxon::MediaOutputUrlRef psd = maxon::ImageSaverClasses::Psd().Create() iferr_return; maxon::MediaSessionRef session; layers.Save(urlResult, psd, maxon::MEDIASESSIONFLAGS::NONE, &session) iferr_return; session.Close() iferr_return; return maxon::OK; }
-
Absolutely wonderful, @ferdinand. I can't thank you and the team at Maxon enough. No worries whatsoever about the delay, you guys make up for it by 10 folds when you come back with a response.
Thank you again. I did indeed notice some similarities with the approach I pasted in my first message. I will try out your proposed implementation later today. I'm very eager to get the alpha channel working.
-
Hey @ferdinand,
Just out of curiosity, I was wondering if there was any hopes for the Image API to handle transparency one day? As you've stated yourself (and as I'm experiencing myself while testing my plugin everyday), iterating pixel by pixel in the goal of splicing channels is very slow when dealing with a lot of semi-transparent images.
Of course, I am gonna stick with the approach you generously proposed as it works, but a faster alternative would eventually be very welcome.
Thank you!
-
Hey @sasha_janvier,
Thank you for reaching out to us. To lead with a direct answer, no there are currently no plans to extend the Image API in that fashion.
And just to be clear, both the Image API and the classic API image types support alphas and multi layer alphas, it is only that the special case of manually assembling a multi layered image with alphas out of loaded files is not something we ever really needed and therefore implemented. BodyPaint that functionality but does it with its own code that is not public.
The only way to make things faster would be to help yourself.
Write things in rows
You could try to do what I did in the working example and mix it with
GetPixelCnt
andSetPixelCnt
(I used it in the Graveyard inConstructMultipassBitmap2
). The idea would be still to write aBaseBitmap
and not aMultipassBitmap
but write data row by row and not pixel by pixel. But since we must blend the pixels ourself, we would still need a bit which blends each row pixel by pixel. Something like this (this is pseudo code):int y; // The current row we are writing. // Buffers for the RGB pixel data of a row. Uchar* bufferA; Uchar* bufferB; Uchar* bufferOut; // Get the data for one row for both images. imageA->GetPixelCnt(0, y, copyWidth, bufferA, COLORBYTES_RGB, COLORMODE::RGB, PIXELCNT::NONE); imageB->GetPixelCnt(0, y, copyWidth, bufferB, COLORBYTES_RGB, COLORMODE::RGB, PIXELCNT::NONE); // Blend both rows into one buffer, BlendBuffers would have to iterate pixel by pixel over both rows. BlendBuffers(bufferA, bufferB, bufferOut); // Write the buffer into the output image. output->SetPixelCnt(0, y, copyWidth, bufferOut, COLORBYTES_ARGB, COLORMODE::ARGB, PIXELCNT::NONE); // Do the same for alphas ...
The problem with this is that doing this will likely eat a good portion of the performance benefit that
GetPixelCnt\SetPixelCnt
yields over the pixel-by-pixel getters and setters because we still callBlendBuffers
.Just use parallelization
Since you say that you have many images to blend, another way out could be simply paralleization. Depending on your image data parallelization might also be necessary. When you have 200 4k images to blend, then writing data in rows offers a theoretical speed-up by the factor of 4096. When you are however trying to blend 1E6 32px images, writing in rows alone will only yield a theoretical speed-up by the factor of 32. The smaller the images and the more you have, the more likely it is that you will need parallelization on top of writing things in rows.
For paralleization you could use jobs (a bit overkill in this case IMHO) or ParallelFor::Dynamic which is our flavor of a parallelized loop.
Poke again at the Image API
After I answered your topic here, I realized a flaw with what I did in the pure Image API code in
ConstructImageTexture
. The problem when I wrote this was thatBaseImageInterface:: AddChildren
only allows you to addImageRef
children to an image, and notImageTextureRef
children. But to load and save files you need this type, i.e., you end up with twoImageTextureRef
you want to insert into aImageTextureRef
(which you cannot). Which is why I had to poke around in the internals of the loaded files withGetFirstChild
. I never spend much time on exploring how theGetFirstChild
image is composed, i.e., simply assumed it had an alpha child. I am pretty sure that when you slam your head hard enough against the wall here, you can make this work. But unfortunately, many things in the Image API have never been documented, so you often have to reverse engineer things.In the end this is all very speculative. I do not really know the context of what you are doing. How many files to blend?, size of the files?, done in one batch or spread over the runtime of a user session?, etc. pp.
Parallelization is often not the best option to make something faster, but it is pretty straight forward. You should however talk to us before you do that unless you feel confident in knowing what you do, as you can also make Cinema 4D slower when you flood its job-queue with too many jobs.
Cheers,
Ferdinand -
As always, thanks a million for your incredibly thorough answer, @ferdinand.
How many files to blend?, size of the files?, done in one batch or spread over the runtime of a user session?, etc. pp.
The exact number of drawn bitmap images depends on the settings set by the user through my plugin's UI, but on average, I would roughly estimate the number of drawn bitmap files to be between 200 and 500. The bitmap's dimensions are set to be 256px by 256px and the drawing process is all done in one batch upon the user triggering the plugin's primary action button.
Thanks for the heads up about the possibility of flooding Cinema 4D's job-queue with too many jobs. This is something I will be very mindful of and aim to prevent in any way I can.
I am extremely grateful for your generous support, but because you've presented multiple methods and concepts I was completely unfamiliar with, you'll have to bear with me as I carefully research and revise them over the week-end before opting for a solution that seems ideal for my specific case.
Thanks again for everything, @ferdinand. I will get back to you with my findings!
Cheers
-
Hey @sasha_janvier,
This seems to be a case where parallelization should work well. What I forgot to say in my last posting is that maxon::ParallelImage could be a good option for you here to replace writing in rows with
Get/SetPixelCnt
.- Use
maxon::ParallelImage
to parallelize reading and writing pixels in your input and output (for a singular result image) at once. - Use
maxon::ParallelFor::Dynamic
or jobs to run multiple pixel writing tasks at once. Make sure to use indeed ::Dynamic and not ::Static and use the default constructor formaxon::Granularity
, i.e., leave it completely up to Cinema 4D to decide how much bandwidth it will give you.
I would recommend starting out with the pixel writing parallelization first, as it will likely yield the largest performance benefit. Maybe this will be then already performant enough for you and you won't need the (likely more complicated) second step with
ParallelFor
.I would also recommend writing a type/struct with which you can more easily express a set of to be blended bitmaps, so that is more easy for you to expose the data to your worker lambdas for
ParallelImage
andParallelFor
.When you run into problems, feel free to ask questions, we know that the maxon API can sometimes be a steep hill to climb for newcomers. When you run into problems with parallelization, I would however have to ask you to open a new thread, as we are reaching here off-topic territories in this thread.
Cheers,
Ferdinand - Use
-
Thank you very kindly, @ferdinand! I had already started exploring the
maxon::ParallelImage
class as I suspected it to be the ideal path forward for my case, so it's great to have my suspicions be validated!I will make sure to start a new thread if I have any questions related to this class.
Thank you very much once again. Your assistance has been indispensable!
Cheers