|
|
Posted by Johnny on 10/12/06 14:07
<Cleverbum@hotmail.com> wrote in message
news:1160653685.087568.173460@h48g2000cwc.googlegroups.com...
> I'm trying to write a script which downloads information from a number
> of websites analyses it and shows some results.
> The problem I'm having is that some sites seem to work perfectly while
> others don't. I know it's to do with the complexity of the site, but
> I've no idea how to fix it in my code.
> At the moment I am just using file_get_contents() to get all of the
> relevant pages, but when I use this on http://www.dontstayin.com it
> just doesn't work!
> when I go there with my browser and view the source it's all lovely
> html, but when I try to grab it file_get_contents() returns
> bool(false)
> what am I doing wrong, and how can I better emulate an actual
> web-browser?
>
curl returns this as it's source:
HTTP/1.1 302 Found
Connection: close
Date: Thu, 12 Oct 2006 14:04:54 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: /pages/home
Cache-Control: private
Content-Type: text/html
<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="/pages/home">here</a>.</h2>
</body></html>
Here's how to get that to a text file using curl
http://us2.php.net/manual/en/ref.curl.php
<?php
$ch = curl_init("http://www.dontstayin.com/");
$fp = fopen("dontstayin_homepage.txt", "w");
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_exec($ch);
curl_close($ch);
fclose($fp);
?>
:^D
[Back to original message]
|