|
|
Posted by Steve on 11/16/07 16:36
"deciacco" <a@a> wrote in message
news:XrmdnVeoe9faJqDanZ2dnUVZ_vqpnZ2d@giganews.com...
> "Jerry Stuckle" <jstucklex@attglobal.net> wrote in message
> news:OaSdnU7vl-_ELqDanZ2dnUVZ_hadnZ2d@comcast.com...
>> deciacco wrote:
>>> "The Natural Philosopher" <a@b.c> wrote in message
>>> news:1195209624.8024.5@proxy00.news.clara.net...
>>>> deciacco wrote:
>>>>> thanks for the reply steve...
>>>>> basically, i want to collect the file information into memory so
>>>>> that I can then do analysis, like compare file times and sizes.
>>>>> it's much faster to do this in memory than to do it from disk.
>>>>> should have mentioned this earlier as you said...
>>>> Why do you care how much memory it takes?
>>>> 1.7MB is not very much.
>>> These days memory is not an issue, but that does not mean we shouldn't
>>> write good, efficient code that utilizes memory well.
>> There is also something known as "premature optimization".
>>> While 1.7MB is not much, that is what is generated when I look at
>>> ~2500 files. I have approximately 175000 files to look at and my
>>> script uses up about 130MB. I was simply wondering if someone out
>>> there with more experience, had a better way of doing this that would
>>> utilize less memory.
>> (Top posting fixed)
>> How are you figuring your 1.7Mb? If you're just looking at how much
>> memory is being used by the process, for instance, there will be a lot of
>> other things in there, also - like your code.
>> 1.7Mb for 2500 files comes out to just under 700 bytes per entry, which
>> seems rather a bit large to me. But it also depends on just how much
>> you're storing in the array (i.e. how long are your path names).
>> I also wonder why you feel a need to store so much info in memory, but
>> I'm sure you have a good reason.
>> P.S. Please don't top post. Thanks.
>
> Jerry...
>
> I use Outlook Express and it does top-posting by default. Didn't realize
> top-posting was bad.
i use oe too. just hit ctrl+end immediately after hitting 'reply group'. a
usenet thread isn't like an email conversation where both parties already
know what was said in the previous coorespondence. top posting in usenet
forces *everyone* to start reading a post from the bottom up. this is
particularly painful when in-line responses are made...you have to not only
read from the bottom up, but find the start of a reponse, read down to see
the in-line response(s), then scroll back up past the start of that post
again.
tons of other reasons. we just ask that you know and try to follow as best
you can what usenet considers uniform/standard netiquette.
> To answer your questions:
<snip>
> "Feel the Need"
> The first post shows you an array of the type of data I store. This array
> gets created for each file and added as an item to another array. In other
> words, an array of arrays. As I mentioned in a fallow-up posting, the
> reason I'm doing this is because I want to do some analysis of file
> information, like comparing file times and sizes from two seperate
> directories. This is much faster in memory than on disk.
ok, for the comparisons...consider speed and memory consumption. if you were
to get a list of file names, your memory consumption would be at its bare
minimum (almost). when doing the comparison, you can vastly improve your
performance *and* maintainability by iterating through the files, getting
the file info, putting that info into a db, and then run queries against the
table. the db will beat your php comparison algorythms any day of the week.
plus, sql is formalized...so everyone will understand how you are making
your comparisons.
the only way to get lower memory consumption would be to, during the process
of listing files, DON'T store the file but immediately put all the
information into the db at that point. that will be the theoretical best
performance and memory utilization combination there can be.
btw, i posted this function in another group and someone asked today what
the hell it does. since it directly relates to what you're doing AND uses
pathinfo and fstat, which i mentioned to you briefly in this thread before,
i thought i'd post this example to help:
==============
<?
function listFiles($path = '.', $extension = array(), $combine = false)
{
$wd = getcwd();
$path .= substr($path, -1) != '/' ? '/' : '';
if (!chdir($path)){ return array(); }
if (!$extension){ $extension = array('*'); }
if (!is_array($extension)){ $extension = array($extension); }
$extensions = '*.{' . implode(',', $extension) . '}';
$files = glob($extensions, GLOB_BRACE);
chdir($wd);
if (!$files){ return array(); }
$list = array();
$path = $combine ? $path : '';
foreach ($files as $file)
{
$list[] = $path . $file;
}
return $list;
}
$files = listFiles('c:/inetpub/wwwroot/images', 'jpg', true);
$images = array();
foreach ($files as $file)
{
$fileInfo = pathinfo($file);
$handle = fopen($file, 'r');
$fileInfo = array_merge($fileInfo, fstat($handle));
fclose($handle);
for ($i = 0; $i < 13; $i++){ unset($fileInfo[$i]); }
echo '<pre>' . print_r($fileInfo, true) . '</pre>';
}
?>
Navigation:
[Reply to this message]
|