|
|
Posted by The Natural Philosopher on 09/01/07 23:14
Gary L. Burnore wrote:
> On Sat, 01 Sep 2007 21:26:46 +0200, "Rik Wasmus"
> <luiheidsgoeroe@hotmail.com> wrote:
>
>> On Sat, 01 Sep 2007 20:39:06 +0200, The Natural Philosopher <a@b.c> wrote:
>>
>>> Rik Wasmus wrote:
>>>> On Sat, 01 Sep 2007 13:13:33 +0200, Rik Wasmus
>>>> <luiheidsgoeroe@hotmail.com> wrote:
>>>>
>>>>> On Sat, 01 Sep 2007 11:42:59 +0200, The Natural Philosopher <a@b.c>
>>>>> wrote:
>>>>>>> If you really want a chunked upload check out the user comments at
>>>>>>> <http://nl3.php.net/manual/en/function.readfile.php>
>>>>>>>
>>>>>> Are you dyslexic, I want to DOWNLOAD. Upload I have done already.
>>>>>
>>>>> bah, 't was late last night... I meant to say download, allthough it
>>>>> depends wether you think usercentric or usercentric which is which :P
>>>> Hmmmz, haven't made a full recovery yet I see, "usercentric or
>>>> servercentric"...
>>>>
>>> its Saturday, Get a bottle of decent spirits and relax. ;-)
>>>
>>> Anyway I have enough onfo op spec out that part of e jobn. Muy necxt
>>> problem is wheher
>> Hehe, taking you own advice? :P
>>
>>> its more efficient to have 5000 files all called 00001, 00002...05000 in
>>> one directory, or whether to split them up over several..and whether to
>>> keep their names and extensions intact, or just be lazy, knowing the
>>> data base knows what they were called.
>> Hmm, depends on the file system, not really an expert there. Jerry would
>> tell you to just serve them up straight from the database, and forget
>> about the filesystem, I'm not so sure :). You can do a little testing
>> offcourse, see what works best.
>
> The more files you store in one directory, the harder the OS has to
> work to list through them.
Ah, but the more subdirectories you have in a file system, the more work...
;-)
I.e. I think that the case where each file is in its own subdirectory is
of similar order to no subdirs at all.
I suspect the answer is that for n files, us the pth root of n as the
number of subdirs, where p is the depth of the subdirs...but a lot
depends on caching algorithms in the directories. AND the way the OS
searches them.
I don't know what a directory entry is these days..256 bytes? well
10,000 of them is only 2.56Mytes or so. Should be well within cache
range. let's say it has to do maybe 1000 machine cycles for every
byte...thats 500K bytes a second searched at 500Mhz..4 seconds for a
linear search to find the last one. Mm. That's significant.
whereas a two level hierearchy? a 100 dirs in the first, and 100 files
in each? 80 milliseconds. Give or take...starting to get into disk
latency times anyway..
well I have no idea how the OS searches the directory, but it seems to
me that a hundred dirs of a hundred files each has to be the way to go.
Shouldn't be that hard either: just use an autoincrement on each tag
record, and every time it gets to modulo one hundred create a new directory.
Navigation:
[Reply to this message]
|