Particles and RenderDocument

peter_horvath

I have a scene with the new 2024.4 particles, like a default Basic Emitter for instance.

I clone the scene and then call RenderDocument() with the NODOCUMENTCLONE flag on a thread, however that does not render the particles.

BaseDocument* renderDocument = static_cast<BaseDocument*>(myDocument->GetClone(COPYFLAGS_DOCUMENT, nullptr));
...
RenderDocument(renderDocument, renderSettings, nullptr, nullptr, bitmap, RENDERFLAGS_EXTERNAL | RENDERFLAGS_BATCHRENDER | RENDERFLAGS_NODOCUMENTCLONE, this->Get());

If RenderDocument() is called without the NODOCUMENTCLONE flag, then the particles are rendered fine.

I assume when RenderDocument clones the scene, it also triggers the particles to be simulated. Can you help me with what's missing to achieve the same behavior?

Have to note, no issue, if the particles are cached.

ferdinand

Hey @peter_horvath,

Thank you for reaching out to us. I had a quick look at RenderDocument and in the case of RENDERFLAGS_NODOCUMENTCLONE not being passed, it does not really do anything obvious which would explain this at first glance (e.g., a BaseDocument::ExecutePasses call). The main thing that differs is that we use an AliasTrans when cloning, which might cause the issue. We are of course also doing a lot of other things but I currently do not see how they could impact particles.

AliasTrans t;
if (!t.Init(maindoc)) goto error;
copy = (BaseDocument*)maindoc->GetClone(flags, &t);

The problem with the code snippet is that we use the internal version of that type and later on a method which is not exposed on the public type. But maybe just emulating using an alias alone will help.

Is there anything special about the context where you call this RenderDocument? I assume RENDERFLAGS_NODOCUMENTCLONE and the other flags are your own symbols. Have you checked if they indeed resolve to RENDERFLAGS::NODOCUMENTCLONE? But last but not least this smells a little bit like a bug to me - that the simulation scene is not correctly initialized when you clone a document or something like that. Will poke one of the particle developers tomorrow if they know anything about this.

Cheers,
Ferdinand

peter_horvath

Hey Ferdinand,

Nothing special about the context. Tried cloning with AliasTrans, but it does not help.

I'm attaching a simple code taken from the manual in case you want to reproduce the issue. The code adds two commands, one calling RenderDocument() with the RENDERFLAGS::NODOCUMENTCLONE flag and one without. You can see, that the one with the flag does not render the particles. Note, that no manual cloning is involved in the example, just loading the document, which might indicate that it's indeed a bug.

test_renderdocument.zip

Thanks,
Peter

ferdinand

Hey @peter_horvath,

first of all once again thanks for your always on point demo projects. That makes it for us that much easier to answer things when we do not have to guess half the things the external developers are doing.

I talked with the particles team and they say this might be related to a bug they are already aware of - rendering particles in the Picture Viewer does not work properly at the moment. I also gave it a quick spin with your demo project and can confirm that this does not work (for me it even does not work in both cases).

The reason why this fails is apparently that the proper preroll is missing although this does not explain everything I encountered. You could combat that with setting the cloned document to frame 0 and then executing the passes up to the point where you want to render. But I did not try that because for me it does not work when you clone or not (and also not in the PV which uses the 'proper' render pipeline), so that seemed a bit pointless. And as said before, I looked at RenderDocument and I saw there no preroll code (but might have overlooked it).

The file particle_test_render.c4d was missing in your project, which is why I had to create my own. Which render engine did you use to render the particles in your tests? Arnold? Because I did use Redshift which might explain differences.

I would recommend opening a bug ticket for this. I have already moved this issue to our bug forum. You can either open the ticket in our bug tracker yourself as an MRD (then you will be able to see the ticket) or I can do it for you (then you won't). If you decide to do it yourself, please add the test file I attached below (or create your own if you have something more fitting). Please also give me a heads-up with the ticket ID so that I can attach the responsible developers to that issue.

Finally, in the ticket or here, a classification of how critical this is for you would be great.

Cheers,
Ferdinand

particle_test_render.c4d

peter_horvath

Hey Ferdinand,

Thanks for the reply. Sorry, I forgot to include the test scene. It was just a simple Basic Emitter and I rendered it with Redshift.

You could combat that with setting the cloned document to frame 0 and then executing the passes up to the point where you want to render.

Yeah, I've tried that actually, but did not solve the problem.

I logged the issue as ITEM#506966 (including the tests scene in the zip this time).

It is not critical for us at the moment, since this is a specific use-case and not using the NODOCUMENTCLONE flag can be a good workaround.

Thank you,
Peter

ferdinand

Hey @peter_horvath,

thank you for the bug report and no need to apologize for the small mistake, as I said, your code examples are usually very helpful - no fluff to the point

Cheers,
Ferdinand

ferdinand

Hey @peter_horvath,

we have touched this with 2024.4.1. For me this now looks like that it is doing what it should do. Can you give it a spin and tell us if it works for you too?

Cheers,
Ferdinand

Code

import c4d
from mxutils import CheckType

doc: c4d.documents.BaseDocument  # The active document

def main() -> None:
    """
    """
    clone: c4d.documents.BaseDocument = CheckType(doc.GetClone(c4d.COPYFLAGS_DOCUMENT, None))

    renderSettings: c4d.documents.RenderData = CheckType(doc.GetActiveRenderData())
    renderSettings[c4d.RDATA_XRES] = 1920
    renderSettings[c4d.RDATA_YRES] = 1080

    bitmap: c4d.bitmaps.BaseBitmap = c4d.bitmaps.BaseBitmap()
    bitmap.Init(1920, 1080)

    c4d.documents.RenderDocument(clone, renderSettings.GetDataInstance(), bitmap, 
                                 c4d.RENDERFLAGS_EXTERNAL | c4d.RENDERFLAGS_BATCHRENDER | 
                                 c4d.RENDERFLAGS_NODOCUMENTCLONE)

    c4d.bitmaps.ShowBitmap(bitmap)
    c4d.EventAdd()
    print ("Finished")

if __name__ == "__main__":
    main()

peter_horvath

Hi Ferdinand,

Thank you for the update. Your python script works indeed, however, weirdly, doing the same thing from c++ still fails. You can check with my sample code attached above.

Edit: I went back to 2024.4.0 and tried the python script there as well and it was working. Not sure what's different between the two calls, but the issue seems to be c++-specific.

Cheers,
Peter

ferdinand

Thanks @peter_horvath, will have a look.

ferdinand

Hello @peter_horvath,

so, I spent some time on this and the underlying issue seems to be that LoadDocument is bugged in the context of particles.

I guess the simulation Team did not or only very hastily looked at your example when they fixed the issue, but a core issue with your example is that you do not preroll the document. I.e., when you just LoadDocument a document which is at frame 10, then you won't see anything in the viewport. A user doing the same thing manually would not see anything either. So you must prerroll the document up to the frame you want to see. RenderDocument seems to do its own prerolling when you omit NODOCUMENTCLONE. But I do not really know where, I looked at the immediate code, there is just some cloning and some internal stuff going on, but no prerolling. I did not waste too much time on hunting down the stack trace here, as this issue is independent of RenderDocument.

I have submitted the issue as "Particle Simulations break when loading documents with LoadDocument" in our bug tracker, find some of the details I have posted there below. Long story short is: LoadDocument does not play nice with particles at the moment, LoadFile does.

Cheers,
Ferdinand

File: particle_test.c4d

import c4d
import os

doc: c4d.documents.BaseDocument  # The currently active document.
op: c4d.BaseObject | None  # The primary selected object in `doc`. Can be `None`.


class ParticleDialog(c4d.gui.GeDialog):
    """
    """
    def CreateLayout(self) -> bool:
        """
        """
        self.SetTitle("Particle Loader")
        self.GroupBorderSpace(5, 5, 5, 5)
        self.AddCheckbox(1000, c4d.BFH_LEFT, 0, 0, "Load with Load File")
        self.AddCustomGui(1001, c4d.CUSTOMGUI_FILENAME, "Scene File", 
                          c4d.BFH_SCALEFIT | c4d.BFH_SCALEFIT, 0, 0, c4d.BaseContainer())
        self.AddEditNumberArrows(1002, c4d.BFH_LEFT, 0, 0)
        self.AddCheckbox(1003, c4d.BFH_LEFT, 0, 0, "Execute as Active Document")
        self.AddButton(1004, c4d.BFH_CENTER, 0, 0, "Load")
        self.SetInt32(1002, 1, 1, 5)
        return True

    def Command(self, id: int, msg: int) -> bool:
        """
        """
        if id == 1004:
            file: str = self.GetString(1001)
            if not file or not os.path.exists(file):
                raise ValueError("Invalid file path")
            
            useLoadFile: bool = self.GetBool(1000)
            passCount: int = self.GetInt32(1002)
            activeDoc: bool = self.GetBool(1003)
            self.LoadParticleDocument(file, activeDoc, passCount, useLoadFile)

        return True

    def LoadParticleDocument(self, path: str, activeDoc: bool, passCount: int, useLoadFile: bool) -> None:
        """
        """
        # Load the document with either LoadDocument or LoadFile.
        if not useLoadFile:
            doc: c4d.documents.BaseDocument = c4d.documents.LoadDocument(
                path, c4d.SCENEFILTER_OBJECTS | c4d.SCENEFILTER_MATERIALS)
            if not doc:
                raise ValueError("Failed to load document")
            
            if activeDoc:
                c4d.documents.InsertBaseDocument(doc)
                c4d.documents.SetActiveDocument(doc)
        else:
            c4d.documents.LoadFile(path)
            doc: c4d.documents.BaseDocument = c4d.documents.GetActiveDocument()
            if not doc:
                raise ValueError("Failed to load document")
            
        doc.SetDocumentName(f"act-{int(activeDoc)}_lfl-{int(useLoadFile)}_cnt-{passCount}")

        # Execute the passes up the frame the document has been loaded with.
        fps: int = doc.GetFps()
        minFrame: int = doc.GetMinTime().GetFrame(fps)
        maxFrame: int = doc.GetTime().GetFrame(fps)

        for frame in range(minFrame, maxFrame + 1):
            doc.SetTime(c4d.BaseTime(frame, fps))
            for i in range(passCount):
                if not doc.ExecutePasses(None, True, True, True, c4d.BUILDFLAGS_0):
                    raise ValueError("Failed to preroll document")
        
        # The case that we use LoadFile and the document is not active, particle will not be 
        # simulated.
        if not useLoadFile and not activeDoc:
            c4d.documents.InsertBaseDocument(doc)
            c4d.documents.SetActiveDocument(doc)

if __name__ == '__main__':
    dlg: ParticleDialog = ParticleDialog()
    dlg.Open(c4d.DLG_TYPE_ASYNC, defaultw=300, default=100)

static maxon::Result<void> Preroll(BaseDocument* const doc)
{
  iferr_scope;
  CheckArgument(doc, "doc"_s, "Document is nullptr."_s);

  const Int32 fps = doc->GetFps();
  const Int32 maxFrame = doc->GetTime().GetFrame(fps);
  const Int32 minTime = doc->GetMinTime().GetFrame(fps);

  // Loop over all frames in the document and execute the passes.
  for (Int32 frame = minTime; frame <= maxFrame; ++frame)
	{
		doc->SetTime(BaseTime(frame, fps));
		if (!doc->ExecutePasses(nullptr, true, true, true, BUILDFLAGS::NONE))
			return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);
	}
  
	return maxon::OK;
}

inline maxon::Result<void> Render(bool clone)
{
   Filename filename = GeGetPluginPath() + Filename("particle_test_render.c4d");

   const SCENEFILTER   flags = SCENEFILTER::OBJECTS | SCENEFILTER::MATERIALS;
   BaseDocument* const loadedDoc = LoadDocument(filename, flags, nullptr);
   if (loadedDoc == nullptr)
      return maxon::IoError(MAXON_SOURCE_LOCATION, MaxonConvert(filename, MAXONCONVERTMODE::NONE), "Could not load document."_s);

   // auto free the loaded document
   AutoFree<BaseDocument> docFree;
   docFree.Assign(loadedDoc);

   // preroll the document
   iferr(Preroll(docFree))
			return maxon::UnexpectedError(MAXON_SOURCE_LOCATION, "Could not preroll document."_s);

   RenderData* const rdata = loadedDoc->GetActiveRenderData();
   if (rdata == nullptr)
      return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);

   BaseContainer renderSettings = rdata->GetDataInstanceRef();

   // just render one frame
   // const BaseTime startFrame = renderSettings.GetTime(RDATA_FRAMEFROM, BaseTime());
   // renderSettings.SetTime(RDATA_FRAMETO, startFrame);
   renderSettings.SetInt32(RDATA_FRAMESEQUENCE, RDATA_FRAMESEQUENCE_CURRENTFRAME);

   // prepare target bitmap
   AutoAlloc<BaseBitmap> bitmap;
   if (bitmap == nullptr)
      return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION);

   const Int32       width = renderSettings.GetInt32(RDATA_XRES);
   const Int32       height = renderSettings.GetInt32(RDATA_YRES);
   const IMAGERESULT imageRes = bitmap->Init(width, height);
   if (imageRes != IMAGERESULT::OK)
      return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION);

   // render the image
   RENDERFLAGS  renderFlags = RENDERFLAGS::EXTERNAL | RENDERFLAGS::BATCHRENDER;
   if (!clone)
      renderFlags = renderFlags | RENDERFLAGS::NODOCUMENTCLONE;

   const RENDERRESULT res = RenderDocument(loadedDoc, renderSettings,
      nullptr, nullptr, bitmap, renderFlags, nullptr);

   if (res != RENDERRESULT::OK)
      return maxon::UnexpectedError(MAXON_SOURCE_LOCATION);

   // show result
   ShowBitmap(bitmap);

   return maxon::OK;
}

peter_horvath

Thanks Ferdinand, that makes sense. Will do the preroll step in my code. Cheers.

ferdinand

Hey @peter_horvath,

Just as an FYI, you are currently limited by the issues lined out in the video, i.e., you will have to use LoadFile at the moment to get a meaningful particle simulation (or let the document be cloned by RenderDocument). The Simulation team has already fixed that issue, it will be delivered with the next major release of Cinema 4D, i.e., 2025.0.0.

In case you want to test that fix, I would recommend moving this discussion over to the beta forum.

Cheers,
Ferdinand