Hey @Visualride-0,
you seem to run into some form of memory leak. That you open that many files in a row on one Cinema 4D instance is of course not really intended. This could either be a l smaller leak in our API that simply manifests more or only at all when you load that many files.
Generally, you could separate out the file iteration into an external script, a shell script or a vanilla Python script if you want, and then simply run c4dpy with your script per file or for batches of 10 or 100 files. Starting and shutting down c4dpy 8000 times in a row will of course take its toll in time, so one would be incentivized to bundle up more files to bring down that number of starts and shutdowns.
I had a look at your code, and apart from your redundant document insertion, I do not see much wrong with it, as it is basically just the loop on fileList. Cinema 4D should have caught your mistake there and prevented you from opening the same file twice, but I have not tested it.
for f in filesList:
# This line will load #f into a BaseDocument, insert it into the opened files, and make it the
# active document. Unlike for BaseDocument.LoadDocument, the following lines are therefore
# redundant.
c4d.documents.LoadFile(f)
# doc = c4d.documents.GetActiveDocument()
# c4d.documents.InsertBaseDocument(doc)
# c4d.documents.SetActiveDocument(doc)
c4d.documents.SaveDocument(c4d.documents.GetActiveDocument(), f, c4d.SAVEDOCUMENTFLAGS_0, c4d.FORMAT_C4DEXPORT)
# This does what it should do, close the active document, and create a new one. But to be
# ultra safe, you could also call c4d.documents.CloseAllDocuments().
c4d.documents.KillDocument(doc)
What escapes mea bit too, is the purpose of what we are doing here, since we just overwrite files with itself. But I guess this is minimized code, so it is only meant to demonstrate the problem. For my own understanding, the files in fileList are all *.c4d files, right?
Fixing your code as shown above will likely not fix the problem, it seems more likely to me that there is a memory leak somewhere. For an intermediate solution, you will probably have to do what I proposed above, split this into batches of Cinema 4D or c4dpy instances to flush your memory by shutting down Cinema 4D.
But we would like to have a look at the problem. Could you provide a sample of the documents, something between 10 or 100 files, so that we can test it with your data? You can send me a cloud storage link via a PM on the forum or via a mail to sdk_support(at)maxon(dot)net. Mail attachments will not work here due to the 50MB limitation.
We strive for Cinema 4D being memory leak free, and if there is a severe leak we are not aware of, we should fix it. But this might take time.
Cheers,
Ferdinand
