Posted by Andy Dingley on 11/14/06 17:42

CptDondo wrote:

> It can take hours for the client to query all of
> the servers in round-robin fashion.

> My solution is to develop intermediate proxy-cache boxes,

This isn't proxying (and so I don't think Squid will help).

If you had a squillion clients querying one server with an identical
request, then you could cache that. What your problem is though is one
client querying lots of endpoints -- effectively many totally separate
requests. You can't cache that - even worse is that you might cache it,
and all "servers" appeared to report the same result!

You might be able to proxy this by setting up proxies (custom-written
but simple) somewhere that had good bandwidth to the servers, then
reported their results in some "denser" fashion to the client. This
isn't a transparent proxy though.

Chances are that you could even do this in-house, maybe even by just
re-writing the client to be multi-threaded. Is it really bandwidth
that's the problem here, or latency?

[Back to original message]