Posted by Phil on 11/01/07 05:04

On Oct 31, 9:58 pm, Jerry Stuckle <jstuck...@attglobal.net> wrote:
> Phil wrote:
> > I should point out that I know this has something to do with the file-
> > size. With a small file it works OK. With a large file it fails to the
> > browser. In this case, a large file is up to 3.5GB of uncompressed
> > ASCII text. There may be up to 100 files to search - I can do that
> > part I think, once I get it to actually search the file! :-) Any help
> > GREATLY appreciated.
>
> > Probably I should mention that I am not a full-time developer ... I'm
> > quite good with shell scripts, REGEX and procedural languages ... I'm
> > a hacker at best with PHP :-) I'm more of a systems person.
>
> > On Oct 31, 5:45 pm, Phil <phillip.corch...@gmail.com> wrote:
> >> I cannot figure why this works fine at the command-line of the linux
> >> server, but will not output anything to the browser, and no errors in
> >> the error_log (syslog). The goal here is to have a page to enter a
> >> search term and grep or zgrep pattern matches in the file (will be a
> >> log file). Probably there is a better way to do this using php, but
> >> this is what I was able to come up with using my limited php-
> >> knowledge. Can anyone help me debug this?
>
> > I should point out that I know this has something to do with the file-
> > size. With a small file it works OK. With a large file it fails to the
> > browser. In this case, a large file is up to 3.5GB of uncompressed
> > ASCII text. There may be up to 100 files to search - I can do that
> > part I think, once I get it to actually search the file! :-) Any help
> > GREATLY appreciated.
> >
> > Probably I should mention that I am not a full-time developer ... I'm
> > quite good with shell scripts, REGEX and procedural languages ... I'm
> > a hacker at best with PHP :-) I'm more of a systems person.
> >
> (Top posting fixed)
>
> Well, some code would help. But are you possibly running out of memory
> and/or execution time? Those are the two main things which cause
> problems with large amounts of data but not small.
>
> Anything in your PHP error log?

there is squat from logs :(

This really seems to be an issue of file size.
Even this simple read/echo for a file of only 9200 lines it fails, say
nothing of my other
files of over 2million lines.

I have max_execution set to 3000 sec, so i don't think that is it. I
don't see too much in
php.ini that controls memory handling. "memory limit" is 32MB and post
limit is 8PM (default).

This is Apache/2.0.52 and PHP 4.3.9 by the way on a dual dual-core P4
3Ghz with 4GB RAM and CentOS 4.4 (32bit) with a 2.6.9-42.0.3.ELsmp
kernel

$file="/var/log/httpd/access_log";
$handle = @fopen("$file", "r");
if ($handle) {
echo "<pre>";
while (!feof($handle)) {
$buffer = fgets($handle, 2048);
print_r ($buffer);
ob_flush();
}
echo "</pre>";
fclose($handle);
}

Sorry for Top Post. See GP for original code snip.

[Back to original message]