Create userdata [SOLVED]

Helper

On 11/02/2014 at 01:37, xxxxxxxx wrote:

Originally posted by xxxxxxxx

Originally posted by xxxxxxxx

Sorry for the long post.
But it's an important concept that I see many people (mostly script writers) get tripped up on.

On the contrary, I am glad you posted this. While I mostly have obeyed the rules you give here, I have now and then not given it enough attention. I think that on a very fast computer, things may work fine, and the same code might crash on slower / busier computers.

The same for me.
Thanks for all the support on this and previous posts!

Regards, Pim

Helper

On 11/02/2014 at 07:37, xxxxxxxx wrote:

To run into the next line of code before the previous line has finished executing is impossible. The
function would need to start a different thread that performs the processing that you would rely
on to be already done and I can garuantee you that EventAdd() does not do this.

Multiple calls to EventAdd() are redundant as the event is only processed when your code has
finished executing and the Cinema 4D main loop was hit again. As stated in the documentation,
EventAdd() adds an event to the event queue and does not immediately process it.

Are you sure removing the first from the two EventAdd() calls fixes your problem?

Originally posted by xxxxxxxx

It takes a long time to open&close HD files. Even virtually. And +/- a few milliseconds can be enough to cause a crash.

Assuming we are talking about an environment where no seperate threads were started without
the proper synchronization, it will not cause a crash whether opening the file takes a millisecond
or a bazillion hours.

-Niklas

Helper

On 11/02/2014 at 07:49, xxxxxxxx wrote:

Originally posted by xxxxxxxx

Assuming we are talking about an environment where no seperate threads were started without
the proper synchronization, it will not cause a crash whether opening the file takes a millisecond
or a bazillion hours.

That is the way I am used to think, too.

Helper

On 11/02/2014 at 08:29, xxxxxxxx wrote:

Based on my research.
It's common knowledge that you don't attempt to open, close, open, close, etc.. HD files in your code while writing to them.
The reason is because of the time it takes to do this can take too long while your code is trying to execute, and lead to crashes.
If this is not correct information. Then there's a whole lot of people out there in the world spreading false information.

When it comes to creating objects, tags, & and materials in C4D.
Based on my experience. It's VERY common for C4D to take long enough to create them that it causes problems with the code following their creation.
Scripts are the worst offenders. Because that's where people tend to stuff multiple create->edit code one after the other. And there's no time for building in them.

Normally what happens when you have proper error checking in place is the object gets created. But the code written after it doesn't do anything. Or doesn't work as expected.
But if the code has very little error checking like the code posted in this thread. C4D will most definitely crash if you try to create an object, and then try to do something to it before it has had the time to finish building the object.

I've run into this issue literally hundreds of times over the years.
I've gotten to the point where I can almost hear C4D groaning whenever I ask it to create an object(even a simple cube).
So any time I'm creating something with code. The first thing that goes through my head is: "Am I giving C4D enough time to create this thing before I try to change it?".

-ScottA

Helper

On 05/11/2014 at 20:17, xxxxxxxx wrote:

Sorry to bring up an ageing thread, but I'm curious to know why creating a new object could cause a crash or errors as mentioned.

If you're running things from the main thread (or any thread for that matter), doesn't the code always run line by line? I thought the time factor would be irrelevant if the code is running on a line-by-line basis, because the next line is not read/run until the current line and it's operation is finished?

WP.

Helper

On 06/11/2014 at 07:44, xxxxxxxx wrote:

When an object (or tag, material, etc..) is created. A bunch of sub routines need to be executed to create it.
One routine executes line-by-line. Then another sub routine executes line by line. etc...
If those sub routines are still being carried out while the code you wrote in your script tries to process the object before those sub routines have finished. Then C4D will crash, because your script code is trying to manipulate an object that hasn't been built yet.
There's no mechanism to make a script stop and wait if those sub routines are not yet finished.
_ Sometimes_ you can get lucky using the sleep() function as a hack. And it will buy you just barely enough time for the sub processes to finish creating the object before your code tries to use it.
Things like creating hair objects and baking can often be cheated to work that way. But it's a very dangerous hack.

To use a real world example.
If you stole an engine from the General Motors assembly line before they got a chance to install all of the spark plugs. And put it in your car. It either wouldn't run at all, or it would run terrible.

Threads and Callbacks can be used in plugins to force a waiting situation. So that you never try to manipulate an object before it's been fully built and cached.
But in C4D plugins. A much simpler way to do that is to use the Message() method to poll for the existence of the object.
If you use the Message() method to poll for an object. The code you write to manipulate that object never gets executed until that object fully exists and is cached. Hence...no crashing.
There's no such mechanisms in a script.

-ScottA

Helper

On 06/11/2014 at 13:46, xxxxxxxx wrote:

Hi ScottA,

The computer science term for what you're describing is a 'race condition'. However, I doubt that the crashes you experience in the context you describe are actually that type of problem.

Just like NiklasR states, it's impossible that any function call that isn't launching a thread needs to be followed by some kind of code execution delay to finish creating a valid object. If reordering your code or adding delays eliminates crashes, you're simply fixing or masking the real problem indirectly. At a conceptual level, this may be limiting how your write your code to a more rigid scope than is actually possible, or making your code slower and more complicated than need be.

However, you are likely exposing one or more real problems, either the SDK docs are lacking vital information about the required order of certain calls, especially when creating new objects, or actual defects in Cinema 4D.

For instance, in the example you give above, you changed the code by removing the first EventAdd() and you moved the InsertObject() call at the end. That lead me to the probable cause. According to the SDK docs, the document takes over ownership of the pointed object when you insert it:

BaseDocument InsertObject description[URL-REMOVED]

The 'ownership' terminology means you lose the ability to modify anything within the null instance after you call InsertObject(). That means everything before and including dd->Set() has to be done beforehand. Although you may be able to read/write the data by retrieving a separate pointer to it through doc, your code will be more complicated than is necessary compared to setting everything up before calling InsertObject().

Important note: calling an object pointer or variable 'null', although technically valid, is dangerously close to the terms 'nullptr' and 'NULL'. It certainly makes discussing the issue at hand harder, as it makes it appear I'm describing some nonsensical use of 'NULL'. A better name for that pointer would be nullObject.

At this point, we're going off topic from 'Create userdata'. For instance, file I/O problems are certainly a separate but important topic. I propose we work together to find the root cause(s) of the crashes you mention and give proper explanations to help everyone in the PluginCafe. ScottA, could you please start a new topic about the problems you believe fixed or hidden by delays and post your before / after code examples you would like me to analyze? I'd greatly appreciate it.

Thanks,

Joey Gaspe
SDK Support Engineer

[URL-REMOVED] @maxon: This section contained a non-resolving link which has been removed.

Helper

On 06/11/2014 at 15:28, xxxxxxxx wrote:

There's two major scenarios of this object crashing that occur very commonly in C4D:
1.)
Crashes when creating objects, then trying to edit the object's attributes in a script (like the code in this thread).

2.)
Crashes when creating objects in plugins, then trying to edit them.
An example of this would be adding an object to the OM in the Execute() method of a tag plugin. Then trying to change the object's attributes in the next line of code.
*Adding new Materials this way will also crash in the same manner.
When inserting objects this way. They are actually None for a few milliseconds.
I've checked this with the console.

-The solution I found to fix the script crashing scenario is to create the object and set up the attributes for it IN MEMORY ONLY.
Then apply them all at once at the end of the script.
I got this idea from reading recommendations on handling HD files. Where you are supposed to prepare your work ahead of time. Then do one single: Open->Write->Close operation. Opposed to doing multiple Open->Write->Close operations several times within a code block.
I've read that opening and closing files too many times. Too quickly. Can cause a crash.

-The solution I found to fix the plugin crashing scenario is to create the object and do a check in the Message() method that the object really does exist. And not run the code in the Execute() method (or any where else) until that check has passed a true condition.

For whatever reason. Time, pointers, or something else.
C4D sometimes needs these "helpers" to insert the objects safely without crashing. And it will not work without them.
Since creating and inserting objects requires a lot of sub steps. It seems very logical to me that the problem is time. Especially since they are returning 'None' for a few milliseconds after being told to build by the code. And when they're given just a little bit of extra time. They return their proper contents and don't crash anymore.

If it's not a time problem. Then I'd be interested in knowing what the real problem is. And why what I'm doing fixes them.
In the end it really doesn't matter that much to me. Because I know how to make the code work without crashing.
But I also don't want to tell people it's a time issue if it's something else.

-ScottA

Helper

On 06/11/2014 at 16:20, xxxxxxxx wrote:

It sounded like you needed to see an example. So I pulled this one from my notes.
The most important thing about this example is that this will crash C4D if used the Execute() method.
Any time I try to create a new object, tag, material in the Execute() method of a tag plugin. It will crash C4D.
But if I do it in the Message() method using a condition that makes C4D pause for a tiny little bit. Or checks to see if the object is truly there and done building. It does not crash.

//This code is used in a tag plugin to automatically add the new tag as the last tag, instead of the first tag  
//When a tag is added by the user the MSG_MENUPREPARE is sent  
//So we can use this message to tell C4D to move the tag after it's been inserted  
  
MyTagData::Message(GeListNode* node, LONG type, void* data);  
{  
  if (type == MSG_MENUPREPARE)  
  {  
   
  BaseTag *myTag = (BaseTag* )node;            //Get the plugin tag  
  if (!myTag) return false;  
   
   
  BaseObject *op = myTag->GetObject();        //Get the object the tag was added to by the user  
  if (!op) return false;   
  
   
  //Find the last tag by looping through the existsing tags  
  BaseTag *last = op->GetFirstTag();  
  if (last)  
  {  
    for (; ; )  
    {  
      if (!last->GetNext()) break;  
      last = last->GetNext();  
    }  
   
      
    myTag->Remove();                        //Remove tag from the 1st position   
      
    op->InsertTag(myTag, last);             //Re-insert tag after last tag  
  }  
  }  
   
  return SUPER::Message(node, type, data)

-ScottA

Helper

On 06/11/2014 at 17:58, xxxxxxxx wrote:

The SDK Documentation clearly states that Execute() is called from a threaded context. And one thing,
besides others, that is forbidden to do from a thread other than the main thread, is scene modifications,
explaining the crashes you experience. See also Important Threading Information > Forbidden Functions[URL-REMOVED].

The Message() function in contrast is called from the main thread, which is why scene modifications are
safe from this context.

Originally posted by xxxxxxxx

[...] The 'ownership' terminology means you lose the ability to modify anything within the null instance after you call InsertObject(). That means everything before and including dd->Set() has to be done beforehand. Although you may be able to read/write the data by retrieving a separate pointer to it through doc, your code will be more complicated than is necessary compared to setting everything up before calling InsertObject(). [...]

That is not exactly true. You are still allowed to modify the object on any level after calling InsertObject().
The document taking the "ownership" over the object just tells you that the objects memory is now
managed by the document, and you do not need to (to be precise, you **must ** not ) free the object by
yourself after you called a function that results in a transfer of ownership.

The opposite direction of this transfer, for instance, would be the BaseList2D::Remove() method. After
you called it, you have to make sure to free the object. The BaseDocument will no longer take care of it.

Note that the ownership of children always belong to their parents. If you deallocate a Null-Object with
a Cube beneath it, the Cube will be freed as well.

Best,
-Niklas

[URL-REMOVED] @maxon: This section contained a non-resolving link which has been removed.

Helper

On 06/11/2014 at 18:53, xxxxxxxx wrote:

Thanks for pointing that out in the docs Nicklas.

-ScottA

Helper

On 10/11/2014 at 15:34, xxxxxxxx wrote:

Hi NiklasR,

Thanks for the clarification. I made it sound stricter because in the specific case given, reordering the code eliminated the crash, and it seemed that inserting the object meant direct modifications could not be done past that point.

As for what ScottA states in his replies:

Please consider there is a lot of false information on the Internet, and some of it may be technically true but poorly explained, leading to misunderstandings.

We have to solve some communication issues here that are making it hard to help dispel some misconceptions. I figured out you're using unconventional terminology for related software development and 3D editor concepts.

For instance, I believe you mean some of the data that represents the 3D elements must be properly created and setup before adding them to the 3D world in Cinema 4D. The 3D world being the data you visualize and manipulate in C4D, as viewed in its viewports, rendered, etc., so all the polygonal shapes, fog effects, etc.

So you state 'in memory' when you mean 'outside of the 3D world', and when you state 'physically create', you mean 'add to the 3D world'. Up to this point, I'd believe we only have a terminology issue, but the concepts are generally correct.

Conceptual issues start with the fact that nothing is 'physically written' when adding objects to the 3D world, rather you are associating your data to the 3D world (doc->InsertObject()), all in the computer's volatile memory space (virtual memory is not a factor at this conceptual level). When you explicitly write your data to a file to commit it to non-volatile memory, you can argue you created a 'physically written' form of your data, as when you turn your computer off, it's still 'represented on a physical area' on your hard disk.

In terms of opening / closing files, you should open the file for as long as you need to access it (read and/or write), and then close it once you know you're done, for efficiency purposes. However, opening / closing a file repeatedly shouldn't crash unless you're not releasing the Windows file handle each time. That will cause a file handle resource leak pile up, eventually causing a crash if the first indication of resource exhaustion is not handled by your code. You could instead leave a file open and write data to it in chunks, closing it at the end, rather than build up the file contents in RAM and then write all in one shot.

More info from Microsoft:
Mark's Blog

Therefore, please consider the concepts relating to resources managed by the OS, in addition to the rules about threading in Cinema 4D, instead of thinking it's always about timing: defective code does not become correct when slowed down or sped up, it just may lead to correct results because the defect is masked by randomly avoiding the conditions that reveal it. You still risk exposing the defect(s) when conditions change (new code, performance variations, etc.). Correct code, on the other hand, should work properly at any speed.

I hope that helps!

Joey Gaspe
SDK Support Engineer

Helper

On 11/11/2014 at 01:41, xxxxxxxx wrote:

Sorry, but I don't agree.
Based on my experiences. There is definitely a short measurable time period (roughly +/- a quarter of a second) between when you tell an object to insert itself into the doc with code. And when the object is inserted, and ready to be accessed with code.
This time period is where it's common to run into crashes if you have code trying to access the object before it's finished being inserted into the document and cached.

This is why I commonly need to set up the object's attributes in memory ahead of time first. And then insert the object at the end of the code.
Or use the messaging system to poll for the object before grabbing it and changing it.

-ScottA