Twitter comes clean on its technical problems

With increasing attention, continuing outages and new funding of $15 million, Twitter is providing more insight into its operations. In a recent blog post, the company shared the technical problems it is facing and how it plans to make Twitter more stable.

Twitter is, fundamentally, a messaging system. Twitter was not architected as a messaging system, however. For expediency's sake, Twitter was built with technologies and practices that are more appropriate to a content management system. Over the last year and a half we've tried to make our system behave like a messaging system as much as possible, but that's introduced a great deal of complexity and unpredictability. When we're in crisis mode, adding more instrumentation to help us navigate the web of interdependencies in our current architecture is often our primary recourse. This is, clearly, not optimal.

Our direction going forward is to replace our existing system, component-by-component, with parts that are designed from the ground up to meet the requirements that have emerged as Twitter has grown. First and foremost amongst those requirements is stability. We're planning for a gradual transition; our existing system will be maintained while new parts are built, and old parts swapped out for new as they're completed. The alternative - scrapping everything for "the big rewrite" - is untenable, particularly given our small (but growing!) engineering and operations team.

Providing this kind of disclosure will go a long way toward appeasing the frustrated fans, as long as the dialogue continues...and Twitter performance shows some ongoing improvements. For a translation of Twitter's outage explanations, see this blog by my colleague Charles Cooper.