Memory Profiler Option Is Super Slow

[gustavo-frizzo]

Hello!

Is it expected or known that austin will be slow when running with -m enabled?

For context, running on my local machine (MAC OS) it runs pretty well, but in other envs (i.e: dev, staging), it took 41 minutes to collect 9 samples (austin-tui).

Example of command used: austin-tui -m -p 3339

The server was running Linux, python 3.9 and the process was a gunicorn around 750 MB.

Also, profiling without the memory option, everything runs fast and without issues.

Thanks!

[P403n1x87]

When running in memory mode, Austin emits samples only when an RSS delta is observed. My hunch is that in this case the gunicorn master process is not allocating/deallocating much memory. By profiling child processes too with the -C option you might be able to see much more activity.

[gustavo-frizzo]

Interesting!

I am almost sure that I was profiling a child process, but I will double check.
Also, I will try the -C option with the master process.

What made me think that it could be slowness is that the austin-tui stayed a long time in the Starting the Austin TUI phase (few minutes).
Btw, the server was not under stress during the profiling.

So, based on what you said, if I have a process eating 2GB of RAM and I fire up austin passing the process' PID, I won't be able to see what part of the application is using that memory? I will be able to see only the memory that gets allocated after austin started to watch it?

Thanks!

[P403n1x87]

What made me think that it could be slowness is that the austin-tui stayed a long time in the Starting the Austin TUI phase (few minutes). Btw, the server was not under stress during the profiling.

The TUI waits for the first sample to start. In wall-time mode this happens immediately, but in CPU and memory mode it might take some time if the process is idle.

So, based on what you said, if I have a process eating 2GB of RAM and I fire up austin passing the process' PID, I won't be able to see what part of the application is using that memory? I will be able to see only the memory that gets allocated after austin started to watch it?

Correct. Also note that Austin doesn't trace memory usage, it simply samples a frame stack when the resident memory changes. Hence guarantees that memory allocations/deallocations are attributed to the right funcion calls decrease at higher sampling intervals. If you need an accurate accounting of memory usage you should consider using a tracing profiler, if you can afford the overhead.

[gustavo-frizzo]

The -C option to capture the child processes of the master did the trick.
At first, I thought I was analyzing a 'busy' child, but that was clearly not the case :)

But unfortunately, as you said, we probably need a different tool, since we would like to know whatever objects are consuming the memory that is already there (tracing tools is still not ideal because of the performance hit).

Would be nice if the python ecosystem had a gc dump tool with an analyzer, similar to what .NET has (If it has, I haven't seen yet 😢) (And sorry for the rant :p)

Thanks for all the help!