Option to wait for c4d to catch up with script ? Memory wise and EventAdd?

[mogh]

Dear Developers,

I have a strange problem. My Polygon merging script with mcomand join runs smoothly, finishing in 14 seconds (500Mb File) and Cinema 4D needs 1Hour on one thread to display me the result !

Is there a way to wait inside the script till c4d alocated memory, placed the new object until the loop continouse?

This idea could be wrong, but hence the script runs till the end I do not know how to handle this behavior.

Any ideas ?
kind regards
mogh

[ferdinand]

Hello @mogh,

thank you for reaching out to us. First of all I am sorry for the long response time, but the weekend and other topics were "in the way"

Regarding your question: This all sounds a bit mysterious. One hour seems pretty long for just processing a scene. Could you provide a full example file which takes this long (you can send your data to be treated confidently to sdk_support(at)maxon.net) ? At least the fragment of your scene you already sent us, is being processed without problems on my machine.

In principle you have might have to decouple your processing (the whole "join"-thingy) from the main thread and then check if the Cinema is still chewing on your data. A Script-Manger script alone will probably not be sufficient for that, you might need a plugin or at least a dialog. But it is really hard to tell from the outside without seeing what is going on.

Cheers,
Ferdinand

[mogh]

hi zipit, no worries,
thank you for your time!

Yes it is misterious, thats why I asked so vague. You can reproduce a heavy data import by putting my object sent to you into a cloner (5x1x5) and convert it (CSO) -
Running the below script takes about 4 seconds to run and about 1Minute plus to display with the same weird finished but not fnished behavior.

Use below script, set your C4D to only have one window otherwise I have status bar issues!
The status bar displays when its fnished with main() the console lags behind but displays 4 secons !

Latest Version, difference to the slim version is mainly with some quality of live code.

by the way, scripts running for 1hour + are no strangers here - we often start them over night ... thats kinda normal the behavior is not
kind regards mogh

#!py3
import c4d, sys, os, math
from c4d import gui
from c4d.documents import GetActiveDocument
#Version 1.5

import cProfile, pstats 
#import time
def profile(func):
    # A simple profiling decorator.
    profiler = cProfile.Profile()

    def wrapper(*args, **kwargs):
        result = None
        try:
            result = profiler.runcall(func, *args, **kwargs)
        except Exception as error:
            raise error
        finally:
            stats = pstats.Stats(profiler)
            # To get the full profile do this.
            stats.strip_dirs().sort_stats("cumtime").print_stats()
            t = round(stats.total_tt, 3)
            #print (f"{func.__name__} took {t} sec.\n")
        return result
    return wrapper

#@profile
def gime_time (milliseconds):
    hours,milliseconds = divmod(milliseconds, 3600000)
    minutes, milliseconds = divmod(milliseconds, 60000)
    seconds = float(milliseconds) / 1000
    s = "%i:%02i:%06.3f" % (hours, minutes, seconds)
    return s

#@profile
def GetNextObject(op):
    if not op: return
    if op.GetDown(): return op.GetDown()
    while op.GetUp() and not op.GetNext():
        op = op.GetUp()
    return op.GetNext()

#@profile
def get_all_objects (op):
    allachsen_list = list()
    all_objects_list = list()
    while op:
        if op.GetName() == 'Achsen-Objekt' or op.GetName() == 'Axis' :
            allachsen_list.append(op)
        all_objects_list.append(op)
        op = GetNextObject(op)
    #return {'all_objects_list' : all_objects}, allachsen
    return all_objects_list, allachsen_list

#@profile
def statusbar (counter, secondcounter):
    #print (secondcounter, int((100*secondcounter)/counter) )
    if int((100*secondcounter)/counter) in [10,20,30,40,50,60,70,80,90,100]:
        #print("Textupdate at: ", int((100*secondcounter)/counter))
        c4d.StatusSetText ('%s - %s Objects are processed.' %(secondcounter, counter))
    c4d.StatusSetBar(int(100*secondcounter/counter)) #statusbar
    #c4d.gui.GeUpdateUI()

def savetorun(checkSelection = False):
    try:
        doc = GetActiveDocument()
    except Exception as e:
        print (e)
        raise TypeError("No Active Document!")
        exit()
        return False

    try:
        op = doc.GetFirstObject()
    except Exception as e:
        print (e)
        c4d.StatusSetText ('No Objects, nothing to do here.')
        raise TypeError("No Objects, nothing to do here.")
        exit()
        return False

    selected = None
    if checkSelection == True:
        selected = doc.GetActiveObject()
        if selected == None:
            c4d.StatusSetText ('No Object/s selected, nothing to do here.')
            print ("No Object/s selected, nothing to do here.")
            exit()
            return False

    return doc, op, selected

def savedoc(doc):
    path = doc.GetDocumentPath()
    name = doc.GetDocumentName()
    name = name[ :-4 ] + "_conectedaxis.c4d"
    filename = os.path.join(path, name)
    saveflags = (c4d.SAVEDOCUMENTFLAGS_DONTADDTORECENTLIST | c4d.SAVEDOCUMENTFLAGS_DIALOGSALLOWED)
    c4d.documents.SaveDocument(doc, filename, saveflags, format=c4d.FORMAT_C4DEXPORT)

#@profile
def JoinCommand(doc, op):
    """ Apply a join command to the object passe as argument
    :param      op: the object to apply current state to object to.
    :type: BaseObject
    :return: the result of the command or raise an error if command failed
    :rtype: BaseObject
    """
    # null = c4d.BaseObject(c4d.Onull)
    # #for o in op.GetChildren:
    # op.InsertUnder(null)
    #settings = c4d.BaseContainer()
    #settings[c4d.MDATA_JOIN_MERGE_SELTAGS] = True
    # bc = settings,
    
    res = c4d.utils.SendModelingCommand(command = c4d.MCOMMAND_JOIN,
                                list = [op],
                                mode = c4d.MODELINGCOMMANDMODE_ALL,                                
                                doc = doc)

    # Cheks if the command didn't failed
    if res is False:
        raise TypeError("return value of Join command is not valid")
    elif res is True:
        print ("Command successful. But no object.")
    elif isinstance(res, list):
        #if c4d.GetC4DVersion() < 21000: res[0].SetAbsPos(c4d.Vector())
        op.Remove()
        return res[0] # Returns the first item containing the object of the list.  ??? GetClone() ???
    
#@profile
def set_point_object_transform(node, transform):
    """Sets the global transform of a point object while keeping its points in place.

    Args:
        node (c4d.PointObject): The point object to move the axis for.
        transform (c4d.Matrix): The new global transform for the object.

    Raises:
        TypeError: When node or transform are not of specified type.
    """
    if (not isinstance(node, c4d.PointObject) or
        not isinstance(transform, c4d.Matrix)):
        msg = f"Illegal argument types: {type(node)}{type(transform)}"
        raise TypeError(msg)

    #mg = node.GetMg()
    # Move the points in the global frame and then into the new frame.
    points = [p * ~transform for p in node.GetAllPoints()]
    # Set the points and stuff ;)
    node.SetAllPoints(points)
    node.Message(c4d.MSG_UPDATE)
    node.SetMg(transform)
    
#@profile
def joinmanagment(n):
    # n "Axis" null will be not alive in a few steps get everything we need from it
    if n.GetUp() :
        parent = n.GetUp()
    else:
        print ("No Parent To Axis Null. Probably not save to run this sript anyway.")
        c4d.StatusClear()
        c4d.StatusSetText ('No Parent found! - Probalby mo CAD import Doc. Script Stopped.')
        exit()
        return False

    newobject = JoinCommand(doc, n) # combine the poly objects

    if not newobject.IsAlive():
        raise TypeError("Object is not alive.")
        return False

    newobject.SetName(str(parent.GetName()))
    newobject.InsertUnder(parent)
    set_point_object_transform(newobject, n.GetMg()) # set points with global matrix from parent n "Axis"

#@profile
def main():
    c4d.CallCommand(13957) # Konsole löschen
    now = c4d.GeGetTimer() # start stopwatch
    # Savetorun: Set True to check for slected objects and get them returned or throw an error, otherwise slected is None.
    doc, op , selected = savetorun()

    c4d.StatusSetSpin()
    all_objects, allachsen = get_all_objects(op) # get two lists

    null_counter = len(allachsen)

    if null_counter == 0: # check if found something to do.
        c4d.StatusClear()
        c4d.StatusSetText ('No Axis Objects found, nothing to do here.')
        print ("No Axis Objects found, nothing to do here.")
        exit()

    counter = len(all_objects)
    secondcounter = 0

    print (counter,' Objects' ,'\n' ,null_counter,' Nulls and their children to connect' ,'\n' ,'----------------------------------------------------' )
    c4d.StatusSetText ('%s Objects are processed.' %(null_counter))

    for n in allachsen:
        secondcounter += 1
        statusbar(null_counter, secondcounter)
        joinmanagment(n)
        #if joinmanagment(n) == False:            break

    print ('----------------------------------------------------')
    print ('END OF SCRIPT %s ms ' %(gime_time(c4d.GeGetTimer() - now))   )
    #savedoc(doc)
    c4d.StatusClear()
    c4d.StatusSetText ('Script finished - C4D needs a while to diplay the new viewport?!')
    c4d.EventAdd()        # update cinema 4d
    return True
    
if __name__=='__main__':
    main()

[ferdinand]

Hi @mogh,

I have posted here a variant of your script. To simplify things, I would ask you to first run this script and see if it does change anything about your problem. Please also try to run the script with #c4d.EventAdd() uncommented in line 155.

Cheers,
Ferdinand

[mogh]

slightly,

the waiting is now in the beginning, with no or sporadic statusbar updates in between and some waiting at the end.

While this seems fine in a generall sense hence the script now takes its time more reasonably, its a bummer that I cannot inform the user about the status of the script in a meaningfull manner.

C4D "hangs and calculates" and is fnished at some point. If I did not know better I would suspect C4D to be crashed and kill it as a user.

I suspect that the c4d.utils.SendModelingCommand does not allow a better solution at this point and only a selfmade C++ version would.

---

Also my hopes for a faster script are shattered, hence the simplified script you posted takes more or less the same time (console reports 10sec but its actually 20seconds) as my 5 year old callcomand script (also 20 seconds) which is a bummer.

I am still a bit lost on how to pursue my "optimzing scripts" endeavor at this point. (which is off topic here).

nevertheless, thank you for your time

kind regards
mogh

[ferdinand]

Hi,

this was just a question, to sort out that my does not script accidentally fix your problem As said in the beginning, if you want more insight into this, I would invite you to provide some test data and the script you are using where you would consider the processing time unreasonable. As already said, you can use the sdk support mail address to provide such data.

Cheers,
Ferdinand

[mogh]

@zipit said in Option to wait for c4d to catch up with script ? Memory wise and EventAdd?:

I would invite you to provide some test data

You can reproduce a heavy data import by putting my object sent to you into a cloner (5x1x5 ... or even more) and convert it (C). No need to send hundreds of megabites through email.

Use your script not my mess if you like.

kind regards
mogh

[ferdinand]

Hi,

so I did that. I took the wheel thingy you did send us, put it in a 5³ cloner, collapsed the cloner and ran the script. It had about 50k objects in the scene and it took me about 65 seconds on my machine (something i7 and 32GB Ram) to chew them down to 151 objects. This is probably not as fast as one would like it to be, but you have to realize that these are a lot of join commands and in this case about 4 million vertices and an equal amount of normals to chew through and reorganize.

I would not consider this to be extremely slow. If you want to to make this more responsive, you will simply have to convert your data in batches. This also could be simply a RAM issue, because you will need a multiples of what the scene geometry takes in RAM. Doing it in batches will also address this, because it will give Cinema and Python's GC time to make room by deallocating unused stuff. But again, one hour seems very long, but without actual test data, this is all just a guessing game.

Cheers
Ferdinand

[mogh]

did sent email with raw data

kind regards
mogh

[ferdinand]

Hi @mogh,

thanks, but I will probably only have a chance to look at it tomorrow. Happy coding and rendering in the mean time

Cheers,
Ferdinand