giampaolo / psutil / 3574

93%

push

travis-ci

<a href="https://github.com/giampaolo/psutil/commit/<a class=hub.com/giampaolo/psutil/commit/6adcca6c9fda5a36e923bfca3591b972e98a0ce5">6adcca6c9<a href="https://github.com/giampaolo/psutil/commit/6adcca6c9fda5a36e923bfca3591b972e98a0ce5">&quot;&gt;Memory leak test: take fluctuations into account (#1757)

Preamble
=======

We have a [memory leak test suite](https://github.com/giampaolo/psutil/blob/</a><a class="double-link" href="https://github.com/giampaolo/psutil/commit/<a class="double-link" href="https://github.com/giampaolo/psutil/commit/e1ea2bccf8aea404dca0f79398f36f37217c45f6">e1ea2bccf</a>">e1ea2bccf</a><a href="https://github.com/giampaolo/psutil/commit/6adcca6c9fda5a36e923bfca3591b972e98a0ce5">/psutil/tests/__init__.py#L897), which calls a function many times and fails if the process memory increased. We do this in order to detect missing `free()` or `Py_DECREF` calls in the C modules. When we do, then we have a memory leak.

The problem
==========

A problem we&#39;ve been having for probably over 10 years, is the false positives. That&#39;s because the memory fluctuates. Sometimes it may increase (or even decrease!) due to how the OS handles memory, the Python&#39;s garbage collector, the fact that RSS is an approximation and who knows what else. So thus far we tried to compensate that by using the following logic:
- warmup (call fun 10 times)
- call the function many times (1000)
- if memory increased before/after calling function 1000 times, then keep calling it for another 3 secs
- if it still increased at all (&gt; 0) then fail

This logic didn&#39;t really solve the problem, as we still had occasional false positives, especially lately on FreeBSD. 

The solution
=========

This PR changes the internal algorithm so that in case of failure (mem &gt; 0 after calling fun() N times) we retry the test for up to 5 times, increasing N (repetitions) each time, so we consider it a failure only if the memory **keeps increasing** between runs. So for instance, here&#39;s a legitimate failure:

```
psutil.tests.test_memory_leaks.TestModuleFunctionsLeaks.test_disk_partitions ... 
Run #1: extra-mem=696.0K, per-call=3.5K, calls=200
Run #2: extra-mem=1.4M, per-call... (continued)

2193 of 2425 relevant lines covered (90.43%)

0.9 hits per line