December 2011
1 post
What Powers Instagram: Hundreds of Instances,...
One of the questions we always get asked at meet-ups and conversations with other engineers is, “what’s your stack?” We thought it would be fun to give a sense of all the systems that power Instagram, at a high-level; you can look forward to more in-depth descriptions of some of these systems in the future. This is how our system has evolved in the just-over-1-year that we’ve been live, and while...
November 2011
2 posts
Instagram Engineering Challenge: The Unshredder
In our office, we have a pretty amazing paper shredder. Seriously, the thing shreds just about anything. It even has a special slot for credit cards (why anyone would want to regularly shred credit cards is beyond me, but I digress…).
One day, after shredding some paper, I thought to myself: shredding paper is a pretty insecure way of destroying important stuff. I figured, it’s a...
Storing hundreds of millions of simple key-value...
When transitioning systems, sometimes you have to build a little scaffolding. At Instagram, we recently had to do just that: for legacy reasons, we need to keep around a mapping of about 300 million photos back to the user ID that created them, in order to know which shard to query (see more info about our sharding setup). While eventually all clients and API applications will have been updated to...
October 2011
1 post
Simplifying EC2 SSH Connections
Here at Instagram, we run our infrastructure on Amazon Web Services, running instances on their Elastic Compute Cloud (EC2). Since we’re often spinning up new machines and changing details of our infrastructure, there’s an ever-growing list of machines that we SSH into.
To authenticate with our instances, we use public key authentication (the recommended way of doing SSH log-ins to...
September 2011
1 post
Sharding & IDs at Instagram
With more than 25 photos & 90 likes every second, we store a lot of data here at Instagram. To make sure all of our important data fits into memory and is available quickly for our users, we’ve begun to shard our data—in other words, place the data in many smaller buckets, each holding a part of the data.
Our application servers run Django with PostgreSQL as our back-end database. Our first...