|
|
Posted by Andy Dingley on 01/10/07 16:33
Ulrich Glumpf wrote:
> As I have the name and the address she could then look them up in the white
> pages but this is a drag as she has to copy and paste the data from my web
> page into the White Pages web page and then submit the form.
Doing this automatically is known as "screen scraping". You'll probably
find lots of useful advice if you search under that term.
There are two difficulties in doing it. It's technically difficult and
it's legally problematic too for copyright issues.
Technically it's awkward because most web sites are designed to be
viewed by humans rather than read by machines. A good
semantically-designed site is easy to scrape, a graphically intensive,
Flash or simply badly-coded site may be impractical to use. There's
also the problem that sites may change their design unpredictably. This
is often sufficient to make your problem start again from scratch. The
best solution I've found to all this (IMHO) is probably to look at
using Python and a library called Beautiful Soup.
Legally there is clear copyright protection (should the site wish) on
anything that resembles a database query. If they aren't actively
encouraging you to do this, they're probably discouraging you and will
have a good legal basis for making you stop. It's also technically
simple to make life hard for you as an automatic query robot and many
"attractive" sites do just this.
If you possibly can do, find a "web service" that offers the same query
service and use that instead. It's far easier than working with raw
HTML.
Navigation:
[Reply to this message]
|