What Twitter can teach about highly reliable and scalable web apps

David M Williams
Friday, 05 December 2008 08:15

IT Industry - Market

Twitter is a popular web 2.0 site gaining huge traction in disproportion to its sheer simplicity. Twitter just asks, "What are you doing?" and nothing more. Despite this it has absolutely millions of page views with a global audience keeping in touch. Here are lessons that any business can take advantage of if they want their web apps to handle massive loads.

Twitter hasn’t always been known for its reliability. Googling “twitter outage” results in much outcry, titled “Another day, another Twitter outage” and “A new kind of Twitter outage” and “The Twitter outage persists” and on and on and on!

The fact so much is said about Twitter outages is in reality a testimony to how popular and influential the service has become for a great many Internet users who depend upon it for their social networking highs.

Every complaint about an outage reflects people who would be hitting the site with page refreshes all day, from any Internet-connected device they can find ranging from desktops to netbooks to PDAs to mobile phones and anything else. I’m sure addicts would Twitter on their car GPS or Kindle e-Book reader if they could.

For corporations Twitter is clearly a site to watch – not necessarily because you want your staff telling the world what they’re doing but because Twitter have had to face the real-world problems of going from zero hits to multi-millions within months. What they have to say about robust web apps is worth listening to.

Twitter themself don’t give a lot away on their main site. The page “How is Twitter built?” simply notes the engineering team work with Ruby on Rails and use Macintoshes except for testing. Ruby on Rails offers rapid feature deployment with a focus on code reuse and templates.

Fortunately, Blaine Cook, Twitter’s lead architect spoke about how he and his team rose to the challenge of a massively popular web application and delivered a presentation on scaling Twitter.

According to the presentation Twitter had, at the time, 600 requests every single second and growing, 180 Rails instances, one single MySQL database server (plus one slave server) and eight Sun X4100 servers.

Blaine says that dealing with scalability is easy, really. Here’s how he achieved it.

CONTINUED