Posted by Bruno Barberi Gnecco on 04/24/07 15:17

ljb wrote:
> brunobgDELETETHIS@users.sourceforge.net wrote:
>
>> I'm using PHP to run a CLI application. It's a script run by cron that
>>parses some HTML files (with DOM XML), and I ended up using PHP to integrate with
>>the rest of the code that already runs the website.
>>
>> The problem is: it's eating more memory than a black hole. It eats the
>>current limit of 256MB set in php.ini, in an application that would hardly
>>consume 4MB if written in C. I don't care if this application takes much longer
>>to run than it would in C, but eating that much memory is not acceptable.
>>
>> So, my question is, how do I find out what is eating that much memory?
>>I'm suspicious of memory leaks, or very stupid garbage collection. Any help?
>
>
> Different sort of problem, but I struggled with a long-running script that
> leaked a bit of memory on each loop, and after a few days/weeks was using
> too much memory. What I had to do was "instrument" the PHP script,
> inserting calls to report current memory usage at frequent intervals. There
> are two ways to do this. memory_get_usage() might be available (depends
> on how PHP was built), but I think it only reports memory used by PHP
> allocators. (Didn't help me, because the leak turned out to be in a loaded
> extension.) The other way (on Linux, for example) is to look at
> /proc/meminfo for total memory usage. Do this often enough in your script,
> and you should be able to narrow down where the memory is being lost.

Thanks. I did this, and the memory is apparently being lost by
the INSERT query. prepare/execute leak *a lot* of memory, while a simple
query() still leaks, but much less.

Thinking it might be related to this bug:
http://bugs.php.net/bug.php?id=39885 (memory leak when doing loads of
INSERT's and there are duplicate key errors), I added a SELECT to avoid
the errors. This had an extraordinary effect: memory consumption started
to decrease, and soon I was using negative memory. When the program ended,
memory_get_usage() was returning '-9415900'.

Since this universe doesn't allow negative memory usage, I
wonder WTF is going on. I'm using MDB2 as DB frontend, BTW, which
may be the culprit here. What frees more memory than it allocated is
a call to query('SELECT ...'). Removing this call leads back to the
endless growing memory problem.

Any ideas to find out what is causing this: mysql, php,
mdb2?

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
Cropp's Law:
The amount of work done varies inversly
with the time spent in the office.

[Back to original message]