Posted by Dikkie Dik on 09/15/05 21:50

nospammmer@gmail.com wrote:
> That's right. It is interesting. This article touches the kind of
> structure I would need for my caching solution.
>
> As you mentionned, I want the class to control the data, no matter if
> it comes from the database or from the cache. But I want it to control
> all the updates, inserts, and deletes also as my data will never be
> modified from outside of the application. The cache will simply be
> updates at the same time as the database.
>
> What I would like to discuss, however, is technically how to index,
> retrieve, update and store large amount of data in shared memory.
>
> Thanks for your help
>
Is the amount of data that large? How come? I'd think that many webpages
are not that large, as web page authors tend to cut the data in quickly
readable chunks. Don't load into memory what you don't need.

If you use collection classes for your data, you effectively createyour
own joins, but in a flexible way. You can decide for each collection
(table) if it is retrieved lazily or greedily.

I noticed that I don't use much SQL JOINs anymore, as the join is
effectively managed by the collection classes. For reading data,
"HANDLER" is my favourite command nowadays. You can't get much faster
than that. The latest thing I am playing with is a "preload" function on
a lazy collection that groups as much requests into one batch (IN clause).
Anyhow, the collections remove many of the duplicate values that you
would otherwise have in an SQL join.

On how to index, the subcollections mentioned in the article can be
useful. If you have a tree-like data structure, subcollections are a
very fast and natural way to organize your data.

Maybe it is useful to give a more detailed description of the kind of
data your web page is managing.

Best regards

[Back to original message]