BaseContainer::SetMemory() question
-
Hi!
I have a custom data type that need to store some raw data blocks, at
CustomDataTypeClass::WriteData()
. Each memory block has an integer id/index and may be read in a different order as saved, so instead of usingHyperFile::WriteMemory()
, I'm usingHyperFile::WriteContainer()
, withBaseContainer::SetMemory()
setting each id/block pair. The API reference forBaseContainer::SetMemory()
says this about the memory pointer:mem: The memory buffer. The container takes the ownership over the memory buffer.
I usually assumes that the BaseContainer creates it's own copy of the data it stores, and owns it. If we store an Int32, it will internally create a GeData with the value. Doesn't it apply to memory too? If I use SetMemory(), will it alloc a copy of the buffer internally in a GeData ank keep it until it's destroyed? That ownership statement got me worried, sounds like the BaseContainer just saves the pointer and takes ownership of the buffer I'm passing to it, instead of allocating new buffer to store. If that's the case, after storing the BaseContainer to the HyperFile, when it's destroyed it will free my original buffer, with bad consequences. Should I allocate a new buffer before storing? In that scenario, will it be stored in the project file?
Same question applies for GeData, it's not clear if
GeData::SetMemory()
will allocate a new block or just take ownership of the buffer. -
Hi Roger thanks for reaching us and sorry for coming late back.
It took a bit of research to find the "way" things are supposed to work and this slowed the whole discussion a bit.With regard to your request, I need to spend a few word on clarifying how
BaseContainer::SetMemory()
andGeData::SetMemory()
work:- both methods expect only memory chunks being allocated on heap: attempting to hand-over those in the stack will simply cause a Cinema crash.
- in BaseContainer::SetMemory() a temporary
GeData
is created on the fly and both the original pointer and size of the memory chunk are passed to instantiate theGeData
. As this temporaryGeData
is copied in theBaseContainer
the temporaryGeData
goes out of scope and gets destroyed with the side effect that also source memory chunk values are lost. - in GeData::SetMemory() a temporary
ByteArray
is created on the fly and the original pointer and size of the memory chunk are passed to instantiate theByteArray
. As this temporaryByteArray
is used to fill theGeData
the temporaryByteArray
goes out of scope and gets destroyed with the side effect that also source memory chunk values are lost.
Considering this scenario, what's written in the documentation is, to a certain extent, misleading, because it's not actually taking the ownership, but rather it's copying the data of the memory chunk in the
BaseContainer
(or theGeData
) and destroying the memory chunk used as source.If this is not intended to happen, you can eventually consider, after using the
SetMemory
in theGeData
or in theBaseContainer
to use the correspondingGetMemory()
(BaseContainer / GeData) and/orGetMemoryAndRelease()
(BaseContainer / GeData) to access the new pointer being stored and use it for any further reason.Last but not least, when using the
GetMemoryAndRelease()
beware of properly freeing the memory being referenced by the pointer returned otherwise you'll end up in memory leak issues.Best, Riccardo
-
Thanks @r_gigante .
So I think I'm doing it right. Passing my original pointer was resulting in a crash, but allocating a copy works fine. This is how I'm inserting a memory block into a BaseContainer that is saved to the HyperFile.
GeData data( DA_BYTEARRAY, DEFAULTVALUE ); if( bc->InsData( dataId, data ) == nullptr || bc->FindIndex( dataId ) == NOTOK ) { return false; } // alloc new memory block auto bufferCopy = TNewMemClear<void*>( Int64( length ) ); CopyMem( buffer, bufferCopy, Int( length ) ); // fill memory block, BaseContainer takes ownership bc->SetMemory( dataId, bufferCopy, Int( length ) ); if( bc->FindIndex( dataId ) == NOTOK ) { return false; }
For reading is simpler, I'm copying the contents from the BaseContainer:
Int bytesRead = 0; auto dataPointer = bc->GetMemory( dataId, bytesRead, nullptr ); CopyMem( dataPointer, buffer, bytesRead );
-
Hi Roger,
what type is
buffer
and how is it allocated? I'm just wondering, if it may be possible to avoid this additional copy tobufferCopy
.Cheers,
Andreas -
I'm saving several data chunks, that my main plugin engine makes care of packing and unpacking. So my
buffer
is avoid*
, it can be anything.If I don't make this copy to pass to SetMemory, C4D crashes. I think correctly, since the plugin engine owns it and will later deallocate.
-
Hello Roger,
my thought was, if you allocated
buffer
properly viaNewMem
(or one of its siblings), then you should be able to use it directly. But as Cinema 4D will do an internal copy and destroy the passed buffer, you'd need to do something like this (pseudocode):SetMemory(buffer) buffer = GetMemory() // to get the pointer to the buffer owned by Cinema // ... and then in the end, if you need to free the memory yourself and can't rely on the memory being destroyed on destruction of the BaseContainer buffer = GetMemoryAndRelease() Free(buffer)
Just thinking and probably you have thought of this option anyway.
Cheers,
Andreas