Simon Frankau's blog

Oh, the Horror: Twitter

First, a couple of disclaimers:

I understand the nature of start-ups. You quickly write rubbish code, because having something running is much more important than anything else. Moreover, the ideas are far more important than the technology details, and if you're successful you'll have plenty of time to rewrite later, right?
This is Twitter Of Old. I'm pretty certain they've hired some people with actual technological skill and have real scalability now.

So, without further ado, I looked at some old presentations on Twitter's implementation and scalability. MY EYES! THE GOGGLES, THEY DO NOTHING! Let us remember that Twitter is a service so reliable that its most memorable image is the Fail Whale that appears every time it goes down...

So, in a presentation in the late 2000s, post-optimisation, they were handling 600 requests per second on 8 Sun boxes. That's 75 messages per box per second. Er, great. We best not talk about e.g. the performance expected out of IRC servers, a decade earlier.

The real killer for me was this quote:

For us, it's really about scaling horizontally - to that end, Rails and Ruby haven't been stumbling blocks, compared to any other language or framework. The performance boosts associated with language would give us a 10-20% improvement, but thanks to architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster than it was in January.

This sounds really cool, until you realise that IN JANUARY, THEY WERE PROCESSING 3 REQUESTS PER SECOND. I've seen shell scripts with better performance, and they're all multi-millionaires now.

Not that I'm bitter. ;)

Posted 2011-08-24.

Full index