The website of popular reality TV series The Biggest Loser crashed this week due to an unexpected spike in traffic and, according to at least some members of the web hosting community, we can expect a higher incidence of high profile site crashes in the immediate future.

Last week, the website of popular Sydney radio station, MMM radio, went down when the station offered downloads of a ring tone during the morning show. And, of course, we here at iTWire had our own bout of the down-time blues when our story about Jon maddog Hall was slash dotted causing a huge 48-hour spike in traffic.

According to Larry Bloch, CEO of Sydney-based hosting services provider,NetRegistry, the combination of broadband and richer media on websites is leading to greatly increased demand on hosting infrastructure. "Sites with highly fluctuating traffic, like TV shows, are particularly sensitive to unanticipated bursts of traffic that cause their dedicated hosting infrastructures to crash," says Bloch. "Within two years you can be sure all these websites will require upgrading to expensive server farm technology or the newly emerging clustered infrastructures."

According to Bloch, individual dedicated servers are highly sensitive to unexpected surges in traffic, especially if the website owner is hosting rich media, sophisticated applications or large files for download.

"The migration path invariably recommended by hosting companies like Webcentral or Telstra is to move towards a server farm with multiple mirrored machines and load balancers distributing requests to this bank of servers," says Bloch. "The future of advanced hosting is without question to leverage huge clustered infrastructures, where customers can share that infrastructure with other clients. The customer data is held securely in a storage area network, but the aspects of the infrastructure that are not security sensitive like routers, servers and load balancers are all shared."

Bloch maintains that the availability of increased bandwidth will eventually for most sites to adopt some type of clustered solution. "A large cluster can sustain enormous peaks in traffic without any performance degradation and shows like The Biggest Loser would keep operating without interruption."