Posted by NC on 02/04/06 18:56

E.T. Grey wrote:
>
> > >I have a (LARGE) set of historical data that I want to keep
> > >on a central server, as several separate files.
> >
> >
> > How large exactly?
>
> At last count, there are about 65,000 distinct files (and increasing)
....
> Each file has the equivalent of approx 1M rows (yes - thats 1 million)
....
> If you multiply the number of rows (on avg) by the number of files -
> you can quickly see why using a db as a repository would be a
> poor design choice.

Sorry, I can't. 65 million records is a manageable database.

> This involves converting the read data to a string first, before
> (possibly) zipping it and sending it. This incurrs overhead (that I
> would like to avoid) on both server and client.

And yet you are willing to convert EVERY BIT of that data when you
search through it...

Cheers,
NC

• Next in forum: Re: php lost variable 0 from the form when i try to save it into mysql database - please help
• Prev in forum: Re: Editing a string to add a \ before a '
• Thread view: Re: Data transfer problem - ideas/solutions wanted (please)

[Reply to this message]