Does Python Always Handle Memory Correctly?

Helper

On 29/05/2013 at 20:34, xxxxxxxx wrote:

Hi,
I have a question about how Python handles memory in our plugins.

I have two plugins that do the same task:
PoseLibrary - Written In C++
PyPoseLibrary -Written in Python

These plugins connect a GeDialog with a special tag plugin that stores images in it. Using the Read(), Write() & CopyTo() methods for storing them in the document's hyperfile.

Both versions work. As far as the user would know.
But the C++ has memory leaks. And the Read, Write() & CopyTo() system in C++ hates it when I do an Insert(new_image) to update one of the BitmapButtons images array with a different one. And throws a debugging error in V.S.
I'm having an absolutely horrendous time getting the C++ version to work without leaking memory and throwing other debugging errors.
And like I said. As a user just using the plugin normally. I would never even know that these errors were happening in the C++ version if I wasn't using it in debugging mode.

So I'm feeling very paranoid about my Python version. And I'm wondering if it too is leaking memory.
When we build plugins in Python. Can I really trust it 100% to handle the memory for me correctly. And in all circumstances?

-ScottA

Helper

On 29/05/2013 at 23:08, xxxxxxxx wrote:

Memory leaks can always be prevented. What does the tracer say? For sure, you have forgotten to free your allocated memory at some point.

Helper

On 30/05/2013 at 01:49, xxxxxxxx wrote:

You simply don't care if there are memory leaks or not in Python, because if there are, it is not your fault
(except in cases where it could be your fault, like messing with ctypes and dlls).

It is however always possible to avoid memory leaks in C++, I have always managed to figure them out
and no plugin of mine has (as far as I can tell from testing) having memory leaks.

-Nik

Helper

On 30/05/2013 at 08:29, xxxxxxxx wrote:

But I do care if I have memory leaks in my python plugins.
I care a lot. Even if it's not my fault.
I need to know if I can rely 100% on Python to handle this correctly for me when I use things that are in the SDK.

@Frank,
In my C++ version. I'm getting memory leaks from the tag plugin.
If I add a BaseBitmap::Free(the specific bitmap here) at the end of the Copy(), and Read() methods. I can fix those leaks. But then it makes the plugin not work correctly.
Specifically. It doesn't save the images to the saved document's hyperfile. So when the .c4d file is opened again. Those images are gone when the GedDialog tries to use them. And I also can't select things with the mouse because it messes up the CopyTo() method.

The other error I get is when I do an insert(render,i) to the array of images in the tag.
I do this remotely from the GeDialog.
The reason for this code is to update an existing Bitmap Button's image by changing a specific image in the tag's image array.
I do this by deleting a specific image in the array(Remove(i)). And inserting a new one in the same array index position(Insert(render,i)). So my GeDailog Buttons always stays in sync with the array of images in the tag.
In simpler words. I find the specific image in the array I want to change. Then I delete it and replace it with a different one. Using the images array index position to do this.
But those three methods Read(), Write(), and CopyTo() don't like it when I do this "Insert()" thing.

The VS output error is:

First-chance exception at 0x0000000140d2ea21 in CINEMA 4D 64 Bit.exe: 0xC0000005: Access violation reading location 0xffffffffffffffff.  
Unhandled exception at 0x0000000140d2ea21 in CINEMA 4D 64 Bit.exe: 0xC0000005: Access violation reading location 0xffffffffffffffff.

And VS also opens a page pointing to this:

void BaseBitmap::Free(BaseBitmap *&bm)  
{  
  C4DOS.Bm->Free(bm);  
  bm=NULL;  
}

I think what's happening is that when I replace an image in the array by using Remove() then Insert(). The Read(),Write, CopyTo() methods are not updating properly to see that change.

If interested. The plugins can be downloaded from my site. With all of the source code: https://sites.google.com/site/scottayersmedia/plugins

The thing that's making my memory so hard to manage it those frack'n three methods.
Read(), Write, & CopyTo(). I hate them.

The reason I made a Python version was because I couldn't fix the debugging errors in the C++ version. But now I'm worried about my Python version too.
I just want to make 100% sure that the Python is truly working properly. And not leaking memory too.

-ScottA

Helper

On 30/05/2013 at 10:00, xxxxxxxx wrote:

with __del__ you can kind of create memory leaks in python. read the gc module in the
python docs. there are also quite some tutorials about the python garbage collector.

Helper

On 30/05/2013 at 10:41, xxxxxxxx wrote:

Originally posted by xxxxxxxx

But I do care if I have memory leaks in my python plugins.
I care a lot. Even if it's not my fault.
I need to know if I can rely 100% on Python to handle this correctly for me when I use things that are in the SDK.

Nobody would be using a language that does not allow to manage memory on your own but
leaks it. Python itself is pretty much memory leak free and if memory leaks appear in you Cinema
plugin, then it is most likely because of a bug in the Python SDK. And those memory leaks are very
rare and can be fixed with a service upgrade without you having to change your code.

-Niklas

Helper

On 30/05/2013 at 11:16, xxxxxxxx wrote:

I normally take it for granted that Python just handles the memory properly.
But since I have two plugin using the same methods Read(), Write(), CopyTo(). And getting horrendous fits from the C++ version in the debugger. I'm now a bit concerned about it.

Like I said. You would never know there was problems in the C++ version if you didn't use a debugger on it.
And since we don't debug our Python plugins. I'm feeling a bit concerned that I'm creating potentially harmful plugins that leak memory.

-ScottA

Helper

On 30/05/2013 at 11:39, xxxxxxxx wrote:

You see memory leaks in the console. If there's a memory leak in your Python plugin, you'll see it in the
console just like a memory leak from a C++ plugin.

-Nik

Helper

On 30/05/2013 at 12:15, xxxxxxxx wrote:

Oh?
I've never seen that before. Do you happen to know when it will show up?

What I mean by that is in C++ the leaks don't show up in the console until you close C4D. They don't show up when the program is running.
And if there is a leak. Instead of the debugger automatically stopping. The console stays open until you manually stop the debugger.
Do you know if it works the same way in Python?

-ScottA

Helper

On 30/05/2013 at 14:17, xxxxxxxx wrote:

The debugger doesn't stop when a memory leak appears, it stops when you do something bad like
accessing an unallocated memory address or memory that you do not have access to. Memory leaks are
only memory leaks at the point the program ends.
Any memory that is not freed at the point the application has teared down is considered a memory leak
and no-one but the programmer (should) know(s) when a memory block *will become* a memory leak at some point
unless you're doing proper reference counting like Python does.

-Nik

Helper

On 30/05/2013 at 16:46, xxxxxxxx wrote:

Is there a way I can create a Python memory leak test script, or plugin, to see this console behavior?

No offense. But I need to see it to believe it.

-ScottA

Edit- I also just remembered that I cannot make Python go into debug mode using the c4d_py_debug.txt file. Which might be why I'm not seeing any messages about memory leaks in my python version.

No matter where I put that text file. Nothing happens in the console that tells me it's in debug mode.
I've tried putting the file in the same folder as the plugin, The User folder, the AppData folder, The Maxon folder in the AppData folder, the python folder in the AppData folder...etc..
Lets just say. I've put the thing everywhere I can think of. And the console never goes into any sort of debugging mode.

Helper

On 02/06/2013 at 11:55, xxxxxxxx wrote:

Can anyone tell me how to use debugging in Python?
I've put "c4d_py_debug.txt" & "c4d_py_verbose.txt" files into my user folder. But nothing new is happening in the console.

What do I do next?

-ScottA