|
|
Posted by Phil on 11/01/07 13:29
On Nov 1, 5:30 am, Jerry Stuckle <jstuck...@attglobal.net> wrote:
> Phil wrote:
> > On Oct 31, 9:58 pm, Jerry Stuckle <jstuck...@attglobal.net> wrote:
> >> Phil wrote:
> >>> I should point out that I know this has something to do with the file-
> >>> size. With a small file it works OK. With a large file it fails to the
> >>> browser. In this case, a large file is up to 3.5GB of uncompressed
> >>> ASCII text. There may be up to 100 files to search - I can do that
> >>> part I think, once I get it to actually search the file! :-) Any help
> >>> GREATLY appreciated.
> >>> Probably I should mention that I am not a full-time developer ... I'm
> >>> quite good with shell scripts, REGEX and procedural languages ... I'm
> >>> a hacker at best with PHP :-) I'm more of a systems person.
> >>> On Oct 31, 5:45 pm, Phil <phillip.corch...@gmail.com> wrote:
> >>>> I cannot figure why this works fine at the command-line of the linux
> >>>> server, but will not output anything to the browser, and no errors in
> >>>> the error_log (syslog). The goal here is to have a page to enter a
> >>>> search term and grep or zgrep pattern matches in the file (will be a
> >>>> log file). Probably there is a better way to do this using php, but
> >>>> this is what I was able to come up with using my limited php-
> >>>> knowledge. Can anyone help me debug this?
> >> > I should point out that I know this has something to do with the file-
> >> > size. With a small file it works OK. With a large file it fails to the
> >> > browser. In this case, a large file is up to 3.5GB of uncompressed
> >> > ASCII text. There may be up to 100 files to search - I can do that
> >> > part I think, once I get it to actually search the file! :-) Any help
> >> > GREATLY appreciated.
>
> >> > Probably I should mention that I am not a full-time developer ... I'm
> >> > quite good with shell scripts, REGEX and procedural languages ... I'm
> >> > a hacker at best with PHP :-) I'm more of a systems person.
>
> >> (Top posting fixed)
>
> >> Well, some code would help. But are you possibly running out of memory
> >> and/or execution time? Those are the two main things which cause
> >> problems with large amounts of data but not small.
>
> >> Anything in your PHP error log?
>
> > there is squat from logs :(
>
> > This really seems to be an issue of file size.
> > Even this simple read/echo for a file of only 9200 lines it fails, say
> > nothing of my other
> > files of over 2million lines.
>
> > I have max_execution set to 3000 sec, so i don't think that is it. I
> > don't see too much in
> > php.ini that controls memory handling. "memory limit" is 32MB and post
> > limit is 8PM (default).
>
> > This is Apache/2.0.52 and PHP 4.3.9 by the way on a dual dual-core P4
> > 3Ghz with 4GB RAM and CentOS 4.4 (32bit) with a 2.6.9-42.0.3.ELsmp
> > kernel
>
> > $file="/var/log/httpd/access_log";
> > $handle = @fopen("$file", "r");
> > if ($handle) {
> > echo "<pre>";
> > while (!feof($handle)) {
> > $buffer = fgets($handle, 2048);
> > print_r ($buffer);
> > ob_flush();
> > }
> > echo "</pre>";
> > fclose($handle);
> > }
>
> > Sorry for Top Post. See GP for original code snip.
>
> OK, in your php.ini file, ensure you have
>
> error_reporting=E_ALL
> display_errors=On
>
> for testing purposes.
>
> If this is a production system, I don't recommend having
> display_errors=On. Rather, I use log_errors=On and set a log file.
>
> P.S. Don't mind the trolls here. Someone is starved for attention.
I already have those options set.
Actually, I partly found the answer, and it was a beginners
mistake ... file permissions, as the logs are root-owned and apache/
php runs as apache - a little 'is_readable' helped me out there. Now I
have to decide if i want to have apache run as root, or modify system
log files. It is 'production' system, but it's an enterprise internal
only single-purpose and with Change Control I can do anything I need
for this.
I'd still be interested in quid-pro-quo on using external grep versus
some internal preg_grep (and how to make that one work ... I've been
stumped so far).
Thanks! phil
Navigation:
[Reply to this message]
|