Way back in time, with the Internet Achive

Tony Austin
Saturday, 18 October 2008 11:45

Opinion and Analysis

Alexa Internet has been crawling the web since 1996, which has resulted in a massive archive" they tell us.

"If you have a web site, and you would like to ensure that it is saved for posterity in the Internet Archive, and you've searched wayback and found no results, you can visit the Alexa's Webmasters page" (and there are a couple of other methods to get your site added to the archive).

They advise that some sites may not be included because the automated crawlers were unaware of their existence at the time of the crawl.

It's also possible that some sites were not archived because they were password protected, blocked by robots.txt, or otherwise inaccessible to their automated systems. Site owners might have also requested that their sites be excluded from the Wayback Machine.

Some sites harder to archive than others. If you look at their collection of archived sites, they say, you will find some broken pages, missing graphics, and some sites that aren't archived at all.

Reasons include unknown sites (not known to the Alexa search engine), JavaScript problems, robot exclusion headers, and orphan pages (pages not linked to from a site's page hierarchy).

Finally, I would be remiss not to provide this link to iTWire's archive at the Wayback Machine (first entry is for 23 March 2005).

Try visiting other sites that you find of interest.

TIP: sometimes you have to prefix the URL with "www", other times you don't. Or you have to try other variations (such as adding the ".au" top level domain extension, as in the case of the iTWire site, rather than using just "itwire.com").