RenderDocument produces different color

bentraje

Hi,

I'm trying the RenderDocument python method to render my file. It works but for some reason it produces a different brightness/contrast compared when using the Render To Picture View command.

You can see the illustration of the problem below. I'm using Redshift when rendering but the problem still applies when just using Standard.

Here's the code I used:

import c4d


def main():
    rd = doc.GetActiveRenderData().GetClone().GetData()

    xRes = int(rd[c4d.RDATA_XRES])
    yRes = int(rd[c4d.RDATA_YRES])

    bmp = c4d.bitmaps.BaseBitmap()
    bmp.Init(x=xRes, y=yRes)

    res = c4d.documents.RenderDocument(doc, rd, bmp, c4d.RENDERFLAGS_EXTERNAL)
    c4d.bitmaps.ShowBitmap(bmp)


if __name__ == '__main__':
    main()

ferdinand

Hello @bentraje,

Thank you for reaching out to us. Your question is missing a key piece of information, which color management mode your render document is in. I assume your render document is either in Basic mode and has Linear Workflow enabled, or you are in OCIO mode.

Fig. I: The color management settings of document, press CTRL + D to open the Attribute Manager on them.

From this follow a few problems.

There are known (and worked on bugs) with OCIO and SDR (8) and HDR (16+) bit depths and the render settings, the problem is quite complicated. In the end, this results in 8 bit images sometimes being treated as 16 bit images and then already pre-transformed 8 bit data being transformed again by the Picture Viewer.
Your problem seems to be related to this, as neither the renderer nor the picture viewer do update the bitmap color profile of the render bitmap when you deviate from the default non-linear sRGB color management. Your script does not make any attempt to rectify this, resulting in different outputs.
You currently lack the tools to cover all cases manually in Python, specifically you are unable to handle the OCIO case.
There is also another bug with the PV causing a crash, when you try to invoke ShowBitmap without having opened the PV once first.

I will talk with the OCIO developer about (2.) as I we were already going to talk about (1.) next week. For (3.) we would have to fix BaseBitmap.SetColorProfile, I have created a task for that, but I am not sure when we will find the time to actually fix it.

Solutions

We could fix RenderDocument. This could either happen en-passant by fixing problem (1.) which is tied to how the bitmaps of a render buffer are handled. Or we could fix the method itself, but this will probably take some time, as at least I would classify this of low priority since there are other ways to deal with this.
Using the render queue or the Teams renderer module should rectify the problem, as you then do not have to manage the bitmap color profiles.
Handle the color profile of the bitmap manually, you can however only do this partially due to being unable to set the OCIO transforms of a bitmap in Python. Find below an example for handling at least the Linear Workflow case.

Cheers,
Ferdinand

Result:

import c4d

doc: c4d.documents.BaseDocument # The active document.

def main():
    """
    """
    rd: c4d.BaseContainer = doc.GetActiveRenderData().GetClone().GetData()
    bmp: c4d.bitmaps.BaseBitmap = c4d.bitmaps.BaseBitmap()
    bmp.Init(int(rd[c4d.RDATA_XRES]), int(rd[c4d.RDATA_YRES]), 32)

    # #bmp currently has the default sRGB 1966-2.1 color profile, i.e., a profile for an SDR color 
    # space with a gamma of ~2.1. When the render document has either "Linear Workflow" or OCIO
    # enabled, this is not the correct render space.

    # Set the color profile to linear when the document in "Linear Workflow" mode.
    if (doc[c4d.DOCUMENT_COLOR_MANAGEMENT] == c4d.DOCUMENT_COLOR_MANAGEMENT_BASIC and
        doc[c4d.DOCUMENT_LINEARWORKFLOW]):
        bmp.SetColorProfile(c4d.bitmaps.ColorProfile.GetDefaultLinearRGB())
    # Set the OCIO color profiles when a document is in OCIO mode. This is currently not possible
    # in Python.
    elif doc[c4d.DOCUMENT_COLOR_MANAGEMENT] == c4d.DOCUMENT_COLOR_MANAGEMENT_OCIO:
        pass

    res: int = c4d.documents.RenderDocument(doc, rd, bmp, c4d.RENDERFLAGS_EXTERNAL)
    if res == c4d.RENDERRESULT_OK:
        c4d.bitmaps.ShowBitmap(bmp)
    
if __name__ == '__main__':
    main()

bentraje

@ferdinand

RE: Your question is missing a key piece of information, which color management mode your render document is in.

Ah gotcha. It's a known bug. Yea I was expecting it to work out of the box since I'm not modifying the document or render data. It's just the same as when running the Shift+R hot key.

Anyhow, your illustration code works as expected. Will close this thread now.

Gregor M

If you try to save this bitmap to .bmp it doesn't retain information about color profile. Is it possible to output the correct image that will open in other apps as it should using basebitmap.Save() function?

import c4d

doc: c4d.documents.BaseDocument

def main():

    rd: c4d.BaseContainer = doc.GetActiveRenderData().GetClone().GetData()
    bmp: c4d.bitmaps.BaseBitmap = c4d.bitmaps.BaseBitmap()
    bmp.Init(int(rd[c4d.RDATA_XRES]), int(rd[c4d.RDATA_YRES]), 32)
    bmp.SetColorProfile(c4d.bitmaps.ColorProfile.GetDefaultLinearRGB())
    res: int = c4d.documents.RenderDocument(doc, rd, bmp, c4d.RENDERFLAGS_EXTERNAL)
    if res == c4d.RENDERRESULT_OK:
        c4d.bitmaps.ShowBitmap(bmp)
        path = "Path/To/Image.bmp"
        bmp.Save(path, c4d.FILTER_BMP, None,)

if __name__ == '__main__':
    main()

ferdinand

Hey @Gregor-M,

Thank you for reaching out to us. As a general note: Generally you should not continue such old threads, as this is usually counter productive. But in this case this is fine, since your question is just a repetition of the OT's question.

I currently do not have that much time, but the issue is related to OCIO. At least that was the case in the past. A BaseBitmap carries color profiles for the principal OCIO color spaces such as 'Display' or 'View Transform'. RenderDocument did not correctly setup these profiles, or more precisely something with copying over these profiles to the final result went wrong. But that should have been fixed, but the recent OCIO changes might have introduced a regression. There is not much what you can do and I will earliest have time next week.

I had a quick look, and when you enable "Scene" as the configuration, the bitmap will look just like the native one. That is because it will then use the display space and view transform of the scene and not the ones attached to the bitmap (so this indeed seems to be the old issue).

I would say this just a cosmetic effect (an admittedly annoying one), but your actual saved bitmap should look the same when you open it in Photoshop or a similar app. Will have a look in a week!

Cheers,
Ferdinand

ferdinand

Hey @Gregor-M,

so, had a look, and it is mostly as I said.

The actual bitmaps saved to disk are the same in your case and correct. The reason why your script produces bitmaps that will look differently in Photoshop or an external image viewer, is because you set the BaseBitmap::COLORPROFILE_INDEX_IMAGE profile to a linear sRGB profile. A manually started rendering will there usually use sRGB2.1. If you wanted to do this correctly, you would have to take the image color profile from the render settings (but since the user will usually have the default value there, this does not really make a difference in most cases):

rd: c4d.BaseContainer = doc.GetActiveRenderData().GetClone().GetData()
bmp: c4d.bitmaps.BaseBitmap = c4d.bitmaps.BaseBitmap()
bmp.Init(int(rd[c4d.RDATA_XRES]), int(rd[c4d.RDATA_YRES]), 32)
bmp.SetColorProfile(rd[c4d.RDATA_IMAGECOLORPROFILE]])

But bitmaps have more than one color profile since OCIO was introduced, they also have a profile for the render space, view transform, and display space. In Python, you cannot yet set these from code yet, as BaseBitmap.Get/SetProfile lacks the index argument there, and you also cannot access these profiles in the document.

After some deep dive into our render pipeline, we found out that this profile data is still incorrectly being copied, the bitmap never gets the correct view transform and display sapce profile assigned.

Independently of this, we implemented OCIO for Python, but it will not yet make it into the upcoming 2025.1 release.
There is currently no way for Python API users to fix this on their side.
This also impacts C++, and there are some hoops to jump through there too. You cannot just get the view transfrom profile for the view transform for the render document and set that. Because that is the actual trasnform chain, i.e., view transform + display space, e.g., "ACES 1.0 SDR-video + sRGB2.1". Set on the bitmap must be the raw view transform profile, e.g., "ACES 1.0 SDR-video. When someone runs into this, and needs C++ code to do this right now, please drop a note here.

Finally, to reiterate:

This only impacts how images are displayed in the PictureViewer when shown via ShowBitmap.
The actual content of the bitmaps is correct.
The OCIO profiles of the bitmap are incorrect at the moment (which impacts the Picture Viewer).
We will fix this, one of our developers is working on it. But we decided not to do the fix in the SDK (RenderDocument) as the core issue lies deeper within the rendering pipeline.

Cheers,
Ferdinand

Gregor M

Thank You for the answer Ferdinand.

Unfortunately I'm sending those files to ComfyUI which has a problem interpreting color profile.
For now I just use Change Render Space and change it to scene-linear and View Transform to Un-tone_mapped.
I'm waiting patiently for this being listed in change log.
Also I'm having problems setting bit depth for those files.

ferdinand

Hello @Gregor-M,

when you have a problem with bit depths, please open a new thread. This is now a bug thread and we should keep it clean.

What I forgot to mention here is that there is one way to sort of fix this right now for you. There is an undocumented flag named RDATA_BAKE_OCIO_VIEW_TRANSFORM_RENDER on the render data (not the same as RDATA_BAKE_OCIO_VIEW_TRANSFORM which is exposed in the UI tto the user). With that you can bake the full transform chain into the bitmap. Such bitmap will then be displayed correctly in the PV. But when you save it to disk, it will be "incorrect", as it got the view and display transform baked into it (which is usually not what you want).

But it could be an option if you only volatiely render documents for displaying purposes only or are okay with rendering things twice (once with the flag for displaying, and once without it for saving). But that is of course very much a hack why we will fix this.

Cheers,
Ferdinand