|
|
Posted by Jerry Stuckle on 11/01/07 20:02
Phil wrote:
> On Nov 1, 5:30 am, Jerry Stuckle <jstuck...@attglobal.net> wrote:
>> Phil wrote:
>>> On Oct 31, 9:58 pm, Jerry Stuckle <jstuck...@attglobal.net> wrote:
>>>> Phil wrote:
>>>>> I should point out that I know this has something to do with the file-
>>>>> size. With a small file it works OK. With a large file it fails to the
>>>>> browser. In this case, a large file is up to 3.5GB of uncompressed
>>>>> ASCII text. There may be up to 100 files to search - I can do that
>>>>> part I think, once I get it to actually search the file! :-) Any help
>>>>> GREATLY appreciated.
>>>>> Probably I should mention that I am not a full-time developer ... I'm
>>>>> quite good with shell scripts, REGEX and procedural languages ... I'm
>>>>> a hacker at best with PHP :-) I'm more of a systems person.
>>>>> On Oct 31, 5:45 pm, Phil <phillip.corch...@gmail.com> wrote:
>>>>>> I cannot figure why this works fine at the command-line of the linux
>>>>>> server, but will not output anything to the browser, and no errors in
>>>>>> the error_log (syslog). The goal here is to have a page to enter a
>>>>>> search term and grep or zgrep pattern matches in the file (will be a
>>>>>> log file). Probably there is a better way to do this using php, but
>>>>>> this is what I was able to come up with using my limited php-
>>>>>> knowledge. Can anyone help me debug this?
>>>> > I should point out that I know this has something to do with the file-
>>>> > size. With a small file it works OK. With a large file it fails to the
>>>> > browser. In this case, a large file is up to 3.5GB of uncompressed
>>>> > ASCII text. There may be up to 100 files to search - I can do that
>>>> > part I think, once I get it to actually search the file! :-) Any help
>>>> > GREATLY appreciated.
>>>> > Probably I should mention that I am not a full-time developer ... I'm
>>>> > quite good with shell scripts, REGEX and procedural languages ... I'm
>>>> > a hacker at best with PHP :-) I'm more of a systems person.
>>>> (Top posting fixed)
>>>> Well, some code would help. But are you possibly running out of memory
>>>> and/or execution time? Those are the two main things which cause
>>>> problems with large amounts of data but not small.
>>>> Anything in your PHP error log?
>>> there is squat from logs :(
>>> This really seems to be an issue of file size.
>>> Even this simple read/echo for a file of only 9200 lines it fails, say
>>> nothing of my other
>>> files of over 2million lines.
>>> I have max_execution set to 3000 sec, so i don't think that is it. I
>>> don't see too much in
>>> php.ini that controls memory handling. "memory limit" is 32MB and post
>>> limit is 8PM (default).
>>> This is Apache/2.0.52 and PHP 4.3.9 by the way on a dual dual-core P4
>>> 3Ghz with 4GB RAM and CentOS 4.4 (32bit) with a 2.6.9-42.0.3.ELsmp
>>> kernel
>>> $file="/var/log/httpd/access_log";
>>> $handle = @fopen("$file", "r");
>>> if ($handle) {
>>> echo "<pre>";
>>> while (!feof($handle)) {
>>> $buffer = fgets($handle, 2048);
>>> print_r ($buffer);
>>> ob_flush();
>>> }
>>> echo "</pre>";
>>> fclose($handle);
>>> }
>>> Sorry for Top Post. See GP for original code snip.
>> OK, in your php.ini file, ensure you have
>>
>> error_reporting=E_ALL
>> display_errors=On
>>
>> for testing purposes.
>>
>> If this is a production system, I don't recommend having
>> display_errors=On. Rather, I use log_errors=On and set a log file.
>>
>> P.S. Don't mind the trolls here. Someone is starved for attention.
>
> I already have those options set.
> Actually, I partly found the answer, and it was a beginners
> mistake ... file permissions, as the logs are root-owned and apache/
> php runs as apache - a little 'is_readable' helped me out there. Now I
> have to decide if i want to have apache run as root, or modify system
> log files. It is 'production' system, but it's an enterprise internal
> only single-purpose and with Change Control I can do anything I need
> for this.
>
Ah, didn't think about the file not being readable. :-)
You NEVER want to run Apache as root. If someone hacks your site, they
have access to EVERYTHING.
> I'd still be interested in quid-pro-quo on using external grep versus
> some internal preg_grep (and how to make that one work ... I've been
> stumped so far).
>
> Thanks! phil
>
>
>
Depends on what you're going to do. If all you're doing is searching an
array, just use preg_grep(). Or, you can read a small file into an
array with file() and use preg_grep().
If you need to do more than one file or a large file, then an external
grep is probably better.
But if I'm searching, I generally have what I'm looking for in a database.
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================
Navigation:
[Reply to this message]
|