Instagram Engineering • Simplifying EC2 SSH Connections

Simplifying EC2 SSH Connections

Here at Instagram, we run our infrastructure on Amazon Web Services, running instances on their Elastic Compute Cloud (EC2). Since we’re often spinning up new machines and changing details of our infrastructure, there’s an ever-growing list of machines that we SSH into.

To authenticate with our instances, we use public key authentication (the recommended way of doing SSH log-ins to EC2 machines), but we need to figure out what host we’re connecting to first. A common way of connecting to an EC2 instance is via it’s public hostname (ec2-x-x-x-x.compute-1.amazonaws.com). However, managing a list of these hostnames once there are several of them is tedious. Because of this, we wrote EC2-SSH, a set of Python scripts that help easily connect to EC2 instances, and that we’re open sourcing for the community’s benefit.

EC2-SSH takes advantage of the EC2 instance tagging. If you’re using the web console, when launching a new instance, Amazon prompts you to provide an optional value for the (pre-filled) “Name” tag. Tags can also be edited using the available EC2 command line tools.

Assuming you’ve already tagged all of your instances with names, using EC2-SSH is as easy as using regular SSH with hostnames— because, behind the scenes, that’s all it’s really doing. Better illustrated with an example: Let’s assume you have an instance tagged with the name “nginx3”; using EC2-SSH you could connect to the instance by typing `ec2-ssh nginx3` into your terminal.

EC2-SSH first calls the Amazon EC2 web service, resolving the tag name (in this case “nginx3”) to the public DNS address. It then substitutes out the tag name with the hostname and sends it, along with any other arguments and parameters, to `/usr/bin/ssh`.

The `ec2-ssh` script is small shell script that calls another Python script, `ec2-host`, that eventually calls `/usr/bin/ssh`. Let’s detail out the process in depth:

The Python script `ec2-host` is distributed in the EC2-SSH Python package and can be used unaccompanied— you might find it rather useful, in fact. Let’s take a look at it’s usage output:

% ec2-host --help

Usage: ec2-host [-k KEY] [-s SECRET] [NAME]
Prints server host name.

  --help                  display this help and exit
  -k, --aws-key KEY       EC2 Key, defaults to ENV[AWS_ACCESS_KEY_ID]
  -s, --aws-secret SECRET EC2 Secret, defaults to ENV[AWS_SECRET_ACCESS_KEY]

By default, with no arguments, `ec2-host` host will return a list of all running EC2 instances and their associated public host names. I often use `ec2-host` this way, combined with grep I use it to filter out and identify a specific instance, or set of instances. Here’s an example:

% ec2-host | grep django
django1    ec2-x-x-x-x.amazonaws.com
django2    ec2-x-x-x-x.amazonaws.com
....

When passing the value of an instance’s “Name” tag as an argument, ec2-host will return the associated public hostname. This is exactly what the `ec2-ssh` shell script does. Here’s an example:

% ec2-host nginx2
ec2-x-x-x-x.compute-1.amazonaws.com

You may be asking how `ec2-host` has access to enumerate over your running EC2 instances, that’s a valid question. Before you start using `ec2-ssh` or `ec2-host` you have to provide your AWS key and secret. You can pass them via command line arguments to `ec2-host` like this:

% ec2-host --aws-key  AKJASKSA1234JDSJ8123 --aws-secret B3JDJRYQ1234QWRHFJ1234AJJDAH1kjd1234

To save time, you can also set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

Putting it all together, if you issue the following command, you’ll be signed in to the instance in your system with the specified name:

% ec2-ssh nginx3

Any arguments after the instance name will be passed into ssh:

% ec2-ssh nginx5 date  
Thu Oct 13 17:25:22 UTC 2011

We’ve made these tools available via PyPy, so you can issue an `easy_install ec2-ssh` or a `pip install ec2-ssh` to install the tools. If you’d like to contribute, you can also fork the code on GitHub, or discuss this post on Hacker News . And if you’re interested in helping us scale our systems, we’d love to hear from you.

Shayne Sweeney, Mobile & Server Engineer