|
|
Posted by SQL Menace on 07/15/07 13:07
On Jul 15, 8:34 am, raylopez99 <raylope...@yahoo.com> wrote:
> On Jul 15, 5:15 am, SQL Menace <denis.g...@gmail.com> wrote:
>
> > Also how will you do one of these queries
> > return all customers who live in LA and have ordered product x between
> > June 2006 and July 2007
>
> > Denis The SQL Menace
>
> You wrote your dB is 300 GB--but how much of this is raw data?
> Perhaps 10%? The other 90% is junk to link the data (I'm guessing).
> If so, you can buy 30 GB of RAM and when x86/Windows supports 64 bit
> better (though I think they already do--Itanium?) you can access this
> 30 GB with no problem. 30 GB RAM costs about $1000. Not expensive.
>
> As for the query: " > return all customers who live in LA and have
> ordered product x between > June 2006 and July 2007"
>
> This is simple using a flat file as I propose: "customers & (order*
> or purchase or buy*) & product x & (DATE T (June 2006 < T < July
> 2007))". Suitable code can be written to make these Boolean operators
> work. Some cleanup might be required to strip out false hits, but
> these false hits are present in regular databases today. Also
> remember right now with RDBMS you expend a tremendous amount of work
> putting data into "orthogonal" databases, via data entry forms. You
> have to pay people to enter the data correctly (even if your program
> rejects bad data entry, you still have to pay people to enter the data
> correctly). You can avoid all of this with a flat file. Just dump
> the raw data into memory and let an inference or search engine index
> the data and make the associations via pointers.
>
> RL
>>Also
> remember right now with RDBMS you expend a tremendous amount of work
> putting data into "orthogonal" databases, via data entry forms
Why, I 'enter' almost all of my data with BCP or BULK INSERT
>>You can avoid all of this with a flat file. Just dump
> the raw data into memory
What about bad data? where are your check constraints?
>>Just dump
> the raw data into memory
Okay I have to import sometimes files that have 100,200, 300 million
rows, they contain cusips, isins, total return, price return (so data
which is decimal) how are you going to import this? BTW they can be in
a variety of formats of course
How will you dump these into memory? You have 300GB of memory on your
server? I don't
>>You wrote your dB is 300 GB--but how much of this is raw data?
> Perhaps 10%? The other 90% is junk to link the data (I'm guessing).
It is all real data, My data goes back to May 1896, we are adding Gigs
as we speak (automated jobs)
Denis The SQL Menace
http://sqlservercode.blogspot.com
http://sqlblog.com/blogs/denis_gobo/default.aspx
Navigation:
[Reply to this message]
|