|
|
Posted by Moot on 04/12/07 20:58
On Apr 12, 11:56 am, "bleen" <blee...@gmail.com> wrote:
> can someone point me in the right direction here..
>
> I have a script (essentially a giant for loop) that takes about 20
> mins to run on my server. I want "split" this process onto a second
> server by doing something like this:
>
> system("/srv1/path/to/php /path/to/script.php ".$arg1." ".$arg2." > /
> dev/null &");
> system("/srv2/path/to/php srv1/path/to/script.php ".$arg1." ".$arg2."
>
> > /dev/null &");
>
> ideas? thoughts? questions? comments? concerns? jokes?
What do you get when you cross a polar bear with a seal?
....
...
..
A polar bear!
And as for your question, it really depends on what you're trying to
do and if that work can be broken down into smaller pieces. You
should go through your script and try to break it out into as many
independent blocks as you can.
You say this is a giant for loop? Does the data need to be worked on
sequentially (ie: does the order of processing matter?). If not, you
can build up a queue of smaller jobs (one for each loop iteration in
your current script?), then offload one job to each server at a time.
When one server finishes, send it another job, etc, until they're all
done. Think of the way those Folding@Home/SETI@Home projects work.
They divide up work into chunks, send each chunk out for processing at
a random computer, and assemble the results into something meaningful
when they come back.
If you can't really split up the work, then you're SOL. I've got a
few of those scripts, myself and there's not much you can do about it.
Navigation:
[Reply to this message]
|