|
|
Posted by Jerry Stuckle on 09/29/07 19:50
qwertycat@googlemail.com wrote:
> On Sep 29, 7:51 pm, Jerry Stuckle <jstuck...@attglobal.net> wrote:
>> Just wondering - why do you need to fork processes, anyway? There's a
>> lot of overhead in doing it, and if they're all CPU bound anyway you
>> aren't going to gain anything (unless you have a potload of CPU's).
>>
>> Forking is good if you have different processes using different
>> resources. But when they have to contend for the same resource,
>> performance often goes down.
>
> Instead of writing a PHP script that downloads 2 million headers from
> a newsgroup in a single connection (which will cause PHP to crash
> anyway as it'll reach 500MB+ memory usage), I thought it would be
> better to launch 4 processes do download it in chunks of 50,000
> headers - with 4 connections to the same NNTP server.
>
Which means you'll be downloading 500MB+ anyway - just in different
processes.
Or you could get some headers and cache them to disk, processing them later.
But which newsgroup has 2M+ headers? Glad I don't have to read that
one! :-)
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================
Navigation:
[Reply to this message]
|