Blaine Cook on Scalability

, Jul 29, 2008

Former Twitter architect Blaine Cook in InfoQ's "virtual panel" on scalability:

"Languages Don't Scale. Frameworks Don't Scale. Architectures Scale."

Scaling is a social problem. The coordination of your business, ops, and engineering people is critical, and if you don't succeed at that, your scaling efforts will fail. If your company doesn't understand the need to coordinate to make growth successful, you will fail. YouTube scaled by getting this social part right. They bought servers, they had a captial plan, they had bandwidth. They chose one thing, did it well, and decided as an organization to scale, it wasn't something that happened magically through the efforts of a genius engineer.

You will fail. Learn quickly, unfail. It's ok. Don't build a service that scales out of the box, because you'll probably learn that your interaction design is wrong, and you're going to have to go back and do it again. In the meantime, someone who didn't build to scale probably got the interaction design right and is now 6 months to a year ahead of you.