Posted by Baron Samedi on 03/12/07 03:58

On 11 Mar, 08:36, Rik <luiheidsgoe...@hotmail.com> wrote:
> Baron Samedi <Papa.Legba....@gmail.com> wrote:
> > I coded up a hit counter, then extended it to see who was reading my
> > blog, by matching IP. The problem is that I am swamped by crawlers.
>
> > How can I detect a human, or a crawler? If I can handle one, I can
> > negate it for the other.
>
> > Should I somehow user $_SERVER['USER_AGENT'] ? or something else?
>
> Nice crawlers: yes.
> Evil crawlers: no, they just copy a valid UA-string.

Well, that's a start. Even if exclude the 'nice' crawlers, it will
help me to sort out. I am drowning in hits and have to eyeball them
anyway. What I am thinking of doing, since more and more people have
static IP addresses nowadays (but not all, sigh), is to just look for
IP known addresses of friends or registered users, since those are
what interest me personally. I am not looking for perfection, just a
feel for who is looking at what.



>
> <http://www.user-agents.org/index.shtml>

That is one long list. Hopefully, I can snarfle some code somewhere
online.

Thanks much for the feedback.


>
> --
> Rik Wasmus
> Posted on Usenet, not any forum you might see this in.
> Ask Smart Questions:http://tinyurl.com/anel

[Back to original message]