Posted by Phil Earnhardt on 02/07/06 20:57

On Tue, 7 Feb 2006 11:07:28 +0000, datasmog@hotmail.com (ray) wrote:

><cmashieldscapting@hotmail.com> wrote:
>
>> How would a Macintosh user best go about the following:
>>
>> --Copying the entire contents of a message board composed of forums.
>> Each forum has threads that must be opened to see the messages, with
>> longer threads divided into pages.
>
>As the pages you describe are generated as a result of queries to a
>database, I'd say you can't. Without the query the page doesn't exist.

If the queries are wired into the HTML links of the pages you wish to
grab, the automated tools to recursively capture an entire website may
be able to pull them down.

Even if you could do that, I'm not sure what you would do once you got
them.

>> I am also interested in copying other website content, such as text,
>> pictures, sound, and moving images.
>
>The contents of a web page are stored in the browser cache on your
>computer, that's how you get to see it.
>There are applications that enable you to download entire static
>websites without actually visiting them. WebGrabber and SiteSucker come
>to mind.

curl ships with OS X. Bring up a terminal window and do

man curl

to see what's available.

--phil

[Back to original message]