BaseLinkArray looses link? (was: Refreshing a nested BaseShader)
-
Hello,
this is related to: https://developers.maxon.net/forum/topic/12823/shader-that-gets-data-from-an-object-refresh/6
My
ShaderData
uses data from anObjectData
to generate its output. It requests the data from the object duringInitRender()
. Using the pattern shown in the above linked thread, theObjectData
sets the Shader dirty (DIRTYFLAGS::DATA
) and also sends anMSG_UPDATE
to it, once the ObjectData has finished computing its data. So, after, every change to theObjectData
's attributes, theShaderData
should be dirty and is told to update.This works flawlessly, if the shader sits directly e.g. in a material's Color channel (even without the
MSG_UPDATE
). However, if I put it into another shader, e.g. a Colorizer, the shader's result is not updated until I change any of the shader's attributes. What can do I have to do with the shader to make sure it is updated, and the material preview is re-rendered, even if it's located somewhere deep down in other shaders in the material?EDIT: I found out what the reason is! Just don't know the solution.
As shown in the above linked thread, I am using a
BaseLinkArray
to keep a list of observers. Initially, this works fine. TheObjectData
holds anObservable
object, and theShaderData
nicely subscribes to it. Callingconst Int32 observerCount = _observers.GetRealCount(doc);
on theBaseLinkArray
returns1
. Setting theShaderData
dirty from within theObjectData
works.But after I put the shader into a Colorizer,
_observers.GetRealCount(doc)
returns0
, while_observers.GetCount()
still returns1
. Therefore, using_observers.GetIndex()
to get the link to the shader from the array returnsnullptr
. It seems theBaseLink
in the array is broken after putting the linked shader into another one.Why? Well, probably because the
BaseShader
that now sits inside the Colorizer is not the same object as the one that was there before, right? But shouldn't that be updated automatically by Cinema, when the shader is placed inside the Colorizer? Isn't that whatBaseLinks
are meant for?Thanks & greetings,
Frank -
-
My first idea would be, to catch a specific message that is maybe sent to the shader when it is being put into the Colorizer.
However, the only message that comes in when this happens, is
1001071
(MSG_MULTI_RENDERNOTIFICATION
). I guess I could use it to update the BaseLink in the observable. But it seems that message comes in very often, simply when clicking through the material editor. Does not sound like a good idea. -
hi,
one question / suggestion :
is there any difference if you change the order of creating your shader ?
for example
add your shader -> add colorizer (that will move your shader inside the colorizer)
add colorizer-> go inside and add your shader.If there's any change, that could mean that your shader is "copied" and not only moved.
I would check if theCopyTo
function is used in the process. That would allow you to update the object's array.I would need a bit of time to recreate an example and see if i can find a solution.
(you can probably test it before me)Cheers,
Manuel -
Hi Manuel,
thanks for the hint! This does in deed sound promising.
And yes, the shader is copied when put into the Colorizer. As far as I remember, almost nothing in Cinema is ever moved, it's all copied.
I addedCopyTo()
to my shader class, and implemented it like this:Bool MyShader::CopyTo(NodeData *dest, GeListNode *snode, GeListNode *dnode, COPYFLAGS flags, AliasTrans *trn) { iferr_scope_handler { GePrint(err.GetMessage()); return false; }; // Get document of destination node // Note: When CopyTo() is called because the shader is moved into another shader (e.g. Colorizer), this is always nullptr. BaseDocument *destDoc = dnode->GetDocument(); // Get container of source node const BaseContainer &dataRef = static_cast<BaseShader*>(snode)->GetDataInstanceRef(); // Get linked object from source node's container BaseObject *linkedObject = dataRef.GetObjectLink(XTERRAINOPERATORSHADER_OPERATORLINK, snode->GetDocument()); // If we have a linked object, and if the destination node has a document if (linkedObject && destDoc) { // Get the node's ObjectData MyObjectData *linkedNodeData = linkedObject->GetNodeData<MyObjectData>(); if (! linkedNodeData) return false; // Get ref to the Observable object MyObservable &observable = linkedNodeData->GetObservableRef(); // Subscribe to shader to observable observable.Subscribe(dnode, destDoc) iferr_return; } return SUPER::CopyTo(dest, snode, dnode, flags, trn); }
It changes nothing. Still doesn't work.
_observers.GetIndex()
still returnsnullptr
, even though_observers.GetCount()
returns1
.What I noticed:
MyShader::CopyTo()
is called numerous times, even if I just navigate through the material manager. When I put my shader into a Colorizer,CopyTo()
is called 10 times!- When
CopyTo()
is called because I move the shader into a Colorizer,dnode->GetDocument()
returnsnullptr
. I tried usingsnode->GetDocument()
instead. Now, after creating my shader, then moving it into a Colorizer, and then changing the Colorizer's gradient once, I have 25(!) subscribers in the observable. And all theBaseLinks
are broken!! Not a single one can be retrieved withGetIndex()
.
I still ask myself why I even have to do all this. Isn't the purpose of BaseLinks to have stable links to nodes even if they're moved and change pointer address? All in all, I'm starting to wonder if a hidden
InExcludeList
inMyObject
's description wouldn't be easier. Incredibly clumsy, yes, but it might just work...Cheers,
Frank -
hi,
About the hidden include list, yes it should work. Note that even if you store the baselink inside the basecontainer of your object, they should be updated by an aliastrans.
That would be a bit better than an hidden in/exclude list.Cheers,
Manuel -
Aha, that's interesting!
So, if I got this right, the problem is that my
BaseLinkArray
is hidden inside myNodeData
where it can't be found by Cinema, hence can't be updated by theAliasTrans
?If it was just one
BaseLink
it would be easy. But I have an arbitrary number ofBaseLinks
in aBaseLinkArray
. Can that somehow be stored in theBaseContainer
? Or is there any other way to make myBaseLinkArray
known to Cinema, so it can be updated? Or a way to update it myself?In the SDK docs, it says about
AliasTrans::Translate()
: "Translates the links in all objects that the translator has come across.". Can I make it "come across" myBaseLinkArray
?@kbar: In the other thread you mentioned that you also implemented an observer pattern. Did you run into this problem, too?
Thanks & greetings,
Frank -
hi,
when you Init your aliastrans, it trying to find all it can everywhere. So every BaseContainer are scanned.
That also include the document's BaseContainer.As you can't say aliastrans where to look, you have to place your data where it can find them.
(but the hidden in/exclude should work)
-
Thank you
Cheers,
Frank -
Just a little follow-up on this:
It would be very much appreciated if there was like a
Message()
coming in, or a callback, or some other way to simply present anyBaseLink
members of theNodeData
to anAliasTrans
.Cheers,
Frank -
It's a never-ending story...
InExcludeList
does not seem to be the answer.I have changed my Observable code to use an InExcludeList in the description of the Observable (the BaseObject that is to be observed). When that object is linked in the Observer (the BaseShader that links to the BaseObject via a LINK description element), I catch
SetDParameter()
, remove the Shader from the previously linked Observable's InExcludeList, and add it to the newly linked Observable's InExcludeList.So far, that works just fine.
However, when nesting the BaseShader in another shader, e.g. a Colorizer, the shader's CopyTo() is called. Here, I am trying to add the destination shader to the Observable's InExcludeList. It does not work.
Bool MyShaderData::CopyTo(NodeData *dest, GeListNode *snode, GeListNode *dnode, COPYFLAGS flags, AliasTrans *trn) { iferr_scope_handler { GePrint(err.GetMessage()); return false; }; // Get everything we need BaseDocument *sourceDoc = snode->GetDocument(); BaseDocument *destDoc = dnode->GetDocument(); // This is always NULL! BaseShader *sourceShader = static_cast<BaseShader*>(snode); BaseShader *destShader = static_cast<BaseShader*>(dnode); // Get linked object from the shader's LINK const BaseContainer &dataRef = sourceShader->GetDataInstanceRef(); BaseObject *linkedObject = dataRef.GetObjectLink(MYSHADER_OPERATORLINK, snode->GetDocument()); if (linkedObject && sourceDoc/*destDoc*/) // Can't check destDoc, as it's always NULL { // Get linked object's Observable instance MyObjectData *linkedObjectData = linkedObject->GetNodeData<MyObjectData>(); if (!linkedObjectData) return false; MyObservable &observable = linkedObjectData->GetObservableRef(); // Add destShader as new observer to the linked object's Observable observable.Subscribe(destShader, linkedObject, MYOBJECT_OBSERVERLIST, sourceDoc) iferr_return; } // Translate (doesn't seem to make a difference) if (trn) trn->Translate(true); return SUPER::CopyTo(dest, snode, dnode, flags, trn); }
I am looking at the InExcludeList in the linked object. After
MyShader::CopyTo()
was called, the list always appears empty. observable.Subscribe() is called, but the new copy of the shader does not show up in it.So, my question remains:
How can I maintain a BaseLink to my shader in an object, after the shader has been copied?Cheers,
FrankADDITION:
Interesting. If I add an InExcludeList to any object, just for a test, and not actually handle it anywhere in my code, it also looses the link to any BaseShader I add to the list, when that BaseShader is then nested in another shader. I just added a useless InExcludeList to some object, linked a Noise shader in it, and then nested the Noise Shader in a Colorizer. Whoops, the InEcludeList is emptySo, it seems, the reason for this not working isn't even in my code. Is this a bug in Cinema, or a general problem, or does the object that holds the InExcludeList have to do something in order to make it work? Catch some message, or something like that?
-
Any news on this one?
A heads-up about how to maintain a reliable list of
BaseLink
s toBaseShader
s in the scene would be most welcome!Greetings,
Frank -
Hi,
@fwilleke80 said in BaseLinkArray looses link? (was: Refreshing a nested BaseShader):
Interesting. If I add an InExcludeList to any object, just for a test, and not actually handle it anywhere in my code, it also looses the link to any BaseShader I add to the list, when that BaseShader is then nested in another shader. I just added a useless InExcludeList to some object, linked a Noise shader in it, and then nested the Noise Shader in a Colorizer. Whoops, the InEcludeList is empty
I tried this in Python and was not able to reproduce it. Which does not mean that it odes not exist, but you might want to provide more details on what leads to this described behaviour. Below you will find a Python script which does what I did read into the quoted paragraph of yours above.
Cheers,
zipit"""Will mimic the steps you described to invoke a faulty InEx behavior. You can run this without any prerequisites in the Script Manager, it will create a null object and a material with three shaders which are linked in the null as described by you. I was not able to reproduce the behavior described by you in the posting, specifically this paragraph: "I just added a useless InExcludeList to some object, linked a Noise shader in it, and then nested the Noise Shader in a Colorizer. Whoops, the InEcludeList is empty [...]" I interpreted your statement as such, as you want to link the shaders BEFORE they are actually a part of the scene graph. At least this is the only way your sentence does make sense to me. But as far as I can tell, everything just seems to work fine? Are you sure, that your shader graph is actually valid? Because if it isn't, obviously also the InEx will fail. """ import c4d def main(): """Will create a setup as described by you. """ # A null with an InEx user data element. null = c4d.BaseList2D(c4d.Onull) bc = c4d.GetCustomDataTypeDefault(c4d.CUSTOMDATATYPE_INEXCLUDE_LIST) bc[c4d.DESC_NAME] = "shaders" null.AddUserData(bc) inex = null[c4d.ID_USERDATA, 1] # A material and three shaders as described by you. material = c4d.BaseList2D(c4d.Mmaterial) shader_colorizer = c4d.BaseList2D(c4d.Xcolorizer) shader_noise = c4d.BaseList2D(c4d.Xnoise) shader_noise.SetName("noise") shader_noise_nested = c4d.BaseList2D(c4d.Xnoise) shader_noise_nested.SetName("noise_nested") # Trying to fulfill your condition of linking the shaders BEFORE they # are bing nested. inex.InsertObject(shader_noise, 0) inex.InsertObject(shader_noise_nested, 0) null[c4d.ID_USERDATA, 1] = inex # Compose the material, i.e. do the nesting. material[c4d.MATERIAL_USE_DIFFUSION] = True material[c4d.MATERIAL_USE_REFLECTION] = False material[c4d.MATERIAL_COLOR_SHADER] = shader_noise material[c4d.MATERIAL_DIFFUSION_SHADER] = shader_colorizer shader_colorizer[c4d.SLA_COLORIZER_TEXTURE] = shader_noise_nested # Insert the nodes into the scene graph. material.InsertShader(shader_colorizer) material.InsertShader(shader_noise) shader_colorizer.InsertShader(shader_noise_nested) doc.InsertObject(null) doc.InsertMaterial(material) # Update stuff material.Message(c4d.MSG_UPDATE) material.Update(True, True) c4d.EventAdd() if __name__=='__main__': main()
-
Hi @zipit,
thank you for investing the time to try this out!
I don't know why it works for you, but it certainly does not for me.
Take a look at this video, where I reproduce the problem completely without writing any custom plugin code. Loosing the
BaseLink
to aBaseShader
when that shader is nested into another shader seems to be default behaviour in Cinema 4D, and I find that highly unsatisfyingI wish the SDK Team would say something about this, or even present a workaround.
Video:
https://frankwilleke.de/misc/C4D_looses_link_to_nested_shader.movIn this video, you can see that, for a test, I added an
IN_EXCLUDE
and aLINK
element to the description of the standard Cube object. Then I create a material, and in that material I create a Noise shader. That shader I link in theIN_EXCLUDE
and theLINK
elements. Works as expected. Then I nest the Noise shader in a Colorizer shader, and both links are gone. Since none of my plugin code is contributing to this behaviour, I don't think that the issue is anywhere in my code.Since both, the
IN_EXCLUDE
and theLINK
element, are usingBaseLink
s, I think theBaseLink
itself is the culprit here.Cheers,
FrankEdit:
Found a way to demonstrate this without even touching any .h/.res files to add elements:Video:
https://frankwilleke.de/misc/C4D_looses_link_to_nested_shader_2.movWhen linking a
BaseShader
inside an XPresso Object Node, the link is also lost when the linked shader is being nested. -
hi,
sorry for the delay (i was on vacation last week), I'm on it.
Cheers,
Manuel -
No worries
-
Hi,
I only briefly scanned this topic, but I have a few thoughts and questions. Simply ignore me, if I talk rubbish as usual...
Does this problem occur with every shader that supports subshaders or only a few? Does it happen with the layer shader as well, for example?
If I remember correctly BaseLinks work only correctly (especially in AliasTrans context), if the linked object is a valid member of a document (e.g. in a NodeData for example via branch. Now, if my mind doesn't trick me, there are certain shaders, which do not properly hold there subshaders in the shader tree. Which caused me all kinds of issues, when trying to initiate shader updates from a NodeData and I ended up with special code for certain shaders (I think, it were mainly those stemming from good ol' Smells Like Almonds shaders), basically extracting the subshaders from their parameters.And another thought, maybe you already tried and I overlooked it in this thread: Does using ForceGetLink() change anything for you, when trying to resolve those links? If I'm not mistaken, C4D also resolves links beyond a document's context with this.
Cheers,
Andreas -
Hi Andreas,
Well, I didn't try ALL the shaders there are, but it does happen with shaders that are not my plugin shader. So far, with all shaders I have tried.
In my example videos, I do have the LINK and IN_EXCLUDE elements on an object in the scene, and in an XPresso node, both loose the link.
ForceGetLink()
does not change anything, unfortunately. It just returnsnullptr
.Cheers,
Frank -
I suspect I won't get an answer here...
I have figured out a way on my own now. It's pretty cumbersome, requires too much code in too many places to post it here, and seems to work flawlessly though it looks fishy.
It is based on keeping BaseLinks to the BaseMaterials that own the BaseShaders. Then, when I need to notify the Observers, I go through all the materials in the list, then iterate all their child BaseShaders until I find the ones that have a link to my Observable object, and then I notify these. Plus, some extra measures to make sure the Observer re-subscribes when specific things happen to it.
It is, like we say in German, from behind through the chest into the eye, but it actually works. Hooray.
Cheers,
Frank -
hi,
never say never. Sorry for the delay, this is a "unusual" case ^^
I did try several options (observable, notification, message etc)
I finished using the function
AddEventNotification
witch is private. (so i had to figure out how it was working)
The shader and the viewport is updated but with a small latency.We can try to improve that solution if you want or you can go with your own.
Below the code for both the ObjectData and the Shader.
The shader need to register to be notified. (this is automatic, nothing to add on the ObjectData's side)
Than you need to check the message in
Message
I feel that the important part is to remove the notification and re/add it. (not sure what it can do if you don't do it ^^)class pc12019_object : public ObjectData { INSTANCEOF(pc12019_object, ObjectData); public: static NodeData* Alloc() { return NewObjClear(pc12019_object); } BaseObject* GetVirtualObjects(BaseObject* op, HierarchyHelp* hh) override { return BaseObject::Alloc(Ocube); } Bool GetDDescription(GeListNode* node, Description* description, DESCFLAGS_DESC& flags) override { if (!description->LoadDescription(Obase)) return false; const DescID* singleid = description->GetSingleDescID(); const DescID cid = DescLevel(OBJ_COLOR, DTYPE_COLOR, 0); if (!singleid || cid.IsPartOf(*singleid, nullptr)) { BaseContainer bc = GetCustomDataTypeDefault(DTYPE_COLOR); bc.SetString(DESC_NAME, "Color"_s); bc.SetData(DESC_GUIOPEN, true); description->SetParameter(cid, bc, DescLevel(ID_OBJECTPROPERTIES)); } flags |= DESCFLAGS_DESC::LOADED; return SUPER::GetDDescription(node, description, flags); } }; class pc12019_shader : public ShaderData { INSTANCEOF(pc12019_shader, ShaderData); public: static NodeData* Alloc() { return NewObjClear(pc12019_shader); } Vector Output(BaseShader* sh, ChannelData* cd) override { return color; } Bool Read(GeListNode* node, HyperFile* hf, Int32 level) override { // need to read the color return true; } Bool Write(GeListNode* node, HyperFile* hf) override { // need to write the color return true; } Bool CopyTo(NodeData* dest, GeListNode* snode, GeListNode* dnode, COPYFLAGS flags, AliasTrans* trn) override { pc12019_shader* destShader = static_cast<pc12019_shader*>(dest); destShader->color = color; return true; } Bool GetDDescription(GeListNode* node, Description* description, DESCFLAGS_DESC& flags) override { if (!description->LoadDescription(Xbase)) return false; const DescID* singleid = description->GetSingleDescID(); DescID cid = DescLevel(SH_OBJECTLINK, DTYPE_BASELISTLINK, 0); if (!singleid || cid.IsPartOf(*singleid, nullptr)) { BaseContainer bc = GetCustomDataTypeDefault(DTYPE_BASELISTLINK); bc.SetString(DESC_NAME, "Link"_s); bc.SetInt32(DESC_CUSTOMGUI, CUSTOMGUI_LINKBOX); BaseContainer ac; ac.SetInt32(Obase, 1); bc.SetContainer(DESC_ACCEPT, ac); description->SetParameter(cid, bc, DescLevel(ID_SHADERPROPERTIES)); } flags |= DESCFLAGS_DESC::LOADED; return SUPER::GetDDescription(node, description, flags); } INITRENDERRESULT InitRender(BaseShader* sh, const InitRenderStruct& irs) override { iferr_scope_handler { err.DiagOutput(); return INITRENDERRESULT::UNKNOWNERROR; }; if (irs.doc == nullptr) return INITRENDERRESULT::OK; BaseContainer* bcShader = sh->GetDataInstance(); if (bcShader == nullptr) return INITRENDERRESULT::UNKNOWNERROR; BaseObject * linkedObject = bcShader->GetObjectLink(SH_OBJECTLINK, irs.doc); if (linkedObject == nullptr) return INITRENDERRESULT::OK; RemoveNotification(sh, linkedObject); AddNotification(sh, linkedObject); GeData data; linkedObject->GetParameter(OBJ_COLOR, data, DESCFLAGS_GET::NONE); color = data.GetVector(); return INITRENDERRESULT::OK; } Bool Init(GeListNode* node) override { BaseMaterial* mat = static_cast<BaseMaterial*>(node); BaseContainer* bc = mat->GetDataInstance(); bc->SetLink(SH_OBJECTLINK, nullptr); return true; } void Free(GeListNode* node) override { GeData data; BaseObject* op = static_cast<BaseObject*>(node); if (op == nullptr) return; BaseDocument* doc = node->GetDocument(); if (doc == nullptr) return; op->GetParameter(SH_OBJECTLINK, data, DESCFLAGS_GET::NONE); BaseList2D* linkedObject = data.GetLink(doc); RemoveNotification(node, linkedObject); } Bool RemoveNotification(GeListNode* node, BaseList2D* linkedObject) { if ((node == nullptr) || (linkedObject == nullptr)) return false; BaseObject* op = static_cast<BaseObject*>(node); if (op == nullptr) return false; BaseDocument* doc = op->GetDocument(); if (doc == nullptr) return false; if (linkedObject->FindEventNotification(doc, op, NOTIFY_EVENT::CACHE)) { return linkedObject->RemoveEventNotification(doc, op, NOTIFY_EVENT::CACHE); } return true; } Bool AddNotification(GeListNode* node, BaseList2D* linkedObject) { BaseObject* op = static_cast<BaseObject*>(node); if (op == nullptr) return false; BaseDocument* doc = op->GetDocument(); if (doc == nullptr) return false; if (!linkedObject->FindEventNotification(doc, op, NOTIFY_EVENT::MESSAGE)) { return linkedObject->AddEventNotification(op, NOTIFY_EVENT::MESSAGE, NOTIFY_EVENT_FLAG::NONE, nullptr); } return true; } Bool Message(GeListNode* node, Int32 type, void* data) override { if (type == MSG_NOTIFY_EVENT) { NotifyEventData *eventData = static_cast<NotifyEventData*>(data); // as we can add several notification we check if we received the right one if (eventData->eventid == NOTIFY_EVENT::MESSAGE) { NotifyEventMsg* eventMsg = static_cast<NotifyEventMsg*>(eventData->event_data); // Checks if the message that the object is fowarding us if the right one. (MSG_DESCRIPTION_POSTSETPARAMETER) if (eventMsg->msg_id == MSG_DESCRIPTION_POSTSETPARAMETER) { DescriptionPostSetValue* dpsv = static_cast<DescriptionPostSetValue*>(eventMsg->msg_data); const Int32 id = dpsv->descid[0][0].id; // Checks if the parameter that have been change is part of the message. if (id == OBJ_COLOR) { // ask the shader to update itself. node->SetDirty(DIRTYFLAGS::ALL); node->Message(MSG_DESCRIPTION_POSTSETPARAMETER); // this will trigger the mat update and the viewport update. (with a small delay) } } } return true; } return SUPER::Message(node, type, data); } private: Vector color = Vector(1); };
Cheers,
Manuel