How To Use Amazon EC2 as Your Desktop

Amazon's Elastic Computing Cloud (EC2) is a service allowing you to rent servers in Amazon datacenters by the hour. Using EC2 for your desktop makes sense if you need a powerful machine with a great Internet connection, colocated with your production servers. There are many benefits to using EC2 for your desktop:

• Free bandwidth and low latency when accessing Amazon Web Services
• Development and Production can run on the same hardware and network
• Excellent network connection
• Public IP address
• SAN-based disk storage with 3x redundancy, blazing performance, easy backups
• Reliable hardware with ECC RAM
• A restart solves all hardware problems since you get a different machine each time
• Free bandwidth to RestBackup™ and other web services built on AWS

And some disadvantages:

• Development requires a decent Internet connection
• Added cost
• Memory upgrades are expensive (cost increases 4x moving from 1.7 GB to 7.5 GB)
• Lack of cheap 64-bit server types

In this article, I describe how I use EC2 as my Linux development desktop. I provide detailed instructions for every step of the setup process. This guide assumes that your client machine is Windows.

Monthly Cost

Amazon offers 32-bit and 64-bit Linux and Windows servers with various amounts of memory and processing power. I use a 32-bit Linux server with 1.7 GB of memory and 1 EC2 Compute Unit (equivalent to a 1.2 GHz Xeon cpu). This server type is called a “Small Instance” or ‘m1.small’. The server boots from a 16 GB chunk of disk space on Amazon Elastic Block Store (EBS). I also rent a static IP address (Elastic IP address) for my server.

Since Amazon charges by the hour, I tend to start up my server when I start my work-day and shut it down in the evening. It takes about 60 seconds to start up and log in. Amazon charges a small fee for my static IP address when it's not attached to a running server. I still save about USD $50 per month by turning the server off on nights and weekends. Here is my average monthly cost breakdown:

• $19.00 for 200 Small Instance hours at $0.095/hour
• $1.76 for 16 GB EBS at $0.11/GB-mo
• $0.03 for 300 MB data transfer in at $0.10/GB
• $0.05 for 300 MB data transfer out at $0.15/GB
• $5.46 for 544 non-attached Elastic IP hours at $0.01/hour
• Total Monthly Charges: $26.26

This works out to about $315 per year for a development machine. Amazon offers discounts for long-term commitments (Reserved Instances), but they are not economical if you're using the machine only during the workday. I would pay $409/year with a 1-year reserved instance and $298/year with a 3-year reserved instance.

See: Amazon EC2 Homepage

Tools Used

FreeNX Solves the Network Latency Problem

X-Windows programs send many sequential requests to the X server to update the screen. When a program is used over the network, each request is delayed by network latency and the whole program feels sluggish. FreeNX solves this problem with a local caching proxy server that can respond to all X requests locally. Your programs run fast and you experience only one round-trip of network latency.

EC2 API Tools

Amazon provides a suite of command-line tools for managing EC2 instances. Use these tools to set up your new instance. Also use them every day from handy 'start-dev' and 'stop-dev' scripts to start and stop your instance.

PuTTY

Use PuTTY to log into your newly created instance, create your user account, and set up FreeNX.

Ubuntu 10.04LTS

This guide assumes that you are running Ubuntu 10.04LTS on your instance. The Ubuntu team has pledged to support this version with bug fixes and security updates until April 2015. They also provide a system image that can boot from EBS.

Set Up Your Windows Client

• Download NX Client for Windows and install with the default options
  • Mirror: nxclient-3.4.0-7.exe
  • sha1 44204f1664120d789239bb20baeddae037d7bb2c
  • md5 3ad96599506f5bc93022df4bab9dd124
• Download PuTTY putty-0.60-installer.exe (mirror) and install with the default options
  • sha1 a91c02a342f9d2670a5081ddb5261e8683dfdbc1
  • md5 46f0615d61d9dad673cc07279ac43ed1
• Download ec2-api-tools.zip and extract as C:\Program Files\ec2-api-tools
  • Mirror: ec2-api-tools.2010-11-15.zip
  • sha1 28e3944391ad197e235a549ecf573007eb0a786d
  • md5 cd0de5244f01a3d402ec2461d05f06aa
  • Amazon EC2 API Tools
  • Command Line Reference
• Download Java SE 6 JRE and install with the default settings

Set Up Your Instance

Choose EC2 Region

EC2 is available in five regions: US West, US East, EU, South-east Asia, and North-east Asia. You should choose the region that has the lowest latency for you. Use www.cloudping.info or traceroute to measure your latency to each region. Due to routing inefficiencies, the AWS region nearest you may not have the lowest latency; try them all.

• US West 204.236.128.1
• US East 216.182.224.1
• Europe 79.125.0.1
• SE Asia 175.41.128.1
• NE Asia 175.41.192.1

For an up-to-date list of addresses, go to the EC2 Developer Forum and look for an announcement titled "Amazon EC2 Public IP Ranges".

EC2 prices are slightly cheaper in the US East (Northern Virginia) region. You don't have to choose the lowest latency region if US East is good enough for you. Emacs and Gnome Terminal are very usable over FreeNX with 160ms of network latency. Eclipse is best with 100ms or less.

Each region has an identifier and API URL:

Region	Identifier	API URL
US West	us-west-1	https://ec2.us-west-1.amazonaws.com/
US East	us-east-1	https://ec2.us-east-1.amazonaws.com/
Europe	eu-west-1	https://ec2.eu-west-1.amazonaws.com/
SE Asia	ap-southeast-1	https://ec2.ap-southeast-1.amazonaws.com/
NE Asia	ap-northeast-1	https://ec2.ap-northeast-1.amazonaws.com/

Get AWS Credentials

• Go to the Amazon EC2 Homepage and click the “Sign Up For Amazon EC2” button and complete the signup process
• Go back to http://aws.amazon.com/, click Account, and then “Security Credentials”
• Click the X.509 Certificates tab and click “Create a new Certificate”
• Click “Download Private Key File” and save the file as Documents\aws\private-key.pem
• Click “Download X.509 Certificate” and save the file as Documents\aws\certificate.pem
• Create a file Documents\aws\ec2env.bat with the following code. Use the correct URL for the region you chose. You may need to use PROGRA~2 in JAVA_HOME if your OS is 64-bit.

  set PATH=%PATH%;"c:\PROGRA~1\ec2-api-tools\bin"
  set EC2_HOME="c:\PROGRA~1\ec2-api-tools"
  set JAVA_HOME="c:\PROGRA~1\Java\jre6"
  set EC2_PRIVATE_KEY=%USERPROFILE%\Documents\aws\private-key.pem
  set EC2_CERT=%USERPROFILE%\Documents\aws\certificate.pem
  set EC2_URL=https://ec2.us-west-1.amazonaws.com/
  

• Open a Command Prompt and run ec2env.bat to load the settings
• Run ec2-describe-regions to test the settings
  

Create SSH Key Pair

An SSH key pair allows you to log into your new instance and create a user account for yourself. EC2 loads the public key into the new server image. You will use the private key on your client.

• ec2-add-keypair sshkeypair
• Copy and paste the lines from “BEGIN RSA PRIVATE KEY” through “END RSA PRIVATE KEY” into the file Documents\aws\sshkeypair.pem. To copy text from the command prompt, first right-click and choose Mark, then highlight the desired text and press Enter.
• Run PuTTYgen from the Start Menu
• On the Conversions menu, choose “Import Key” and open sshkeypair.pem
• Change the comment to “sshkeypair“
• Enter a good passphrase
• Click the “Save private key” button and save it as Documents\aws\sshkeypair.ppk
  
• Close PuTTY Key Generator
• Delete sshkeypair.pem for safety
• Double-click on sshkeypair.ppk and enter the passphrase. This will start the PuTTY Authentication Agent (pageant) and load the private key.
  

Create Your Instance

• Choose your AMI (Amazon Machine Image)
  • US West: ami-cb97c68e
  • US East: ami-714ba518
  • Europe: ami-a94d67dd
  • SE Asia: ami-2c354b7e
  • NE Asia: ami-5c0fa45d
  These images are published by Canonical Ltd, the commercial sponsor of Ubuntu Linux. They are all 32-bit Ubuntu 10.04 LTS images that boot from EBS. To find updated AMIs, go to http://alestic.com/ and click the tabs under the title “Ubuntu and Debian AMIs for Amazon EC2”.
• Create and start your instance with the following command. Replace ami-FFFFFFFF with the proper AMI identifier. The command may take 30 seconds or more to complete.
  ec2-run-instances --group default --key sshkeypair --block-device-mapping "/dev/sda1=:16:off" --instance-initiated-shutdown-behavior stop --disable-api-termination ami-FFFFFFFF
  
  What the options mean:
  • “--group default” puts this server in the default security group
  • “--key sshkeypair” loads your public key into the new server so you can log in
  • “--block-device-mapping "/dev/sda1=:16:off"” maps /dev/sda1 to a new 16 GB EBS volume. EC2 will not automatically delete the volume when you terminate your instance.
  • “--instance-initiated-shutdown-behavior stop” configures EC2 to stop your instance when you shutdown the OS, preserving the instance ID and config for easy restarting. Without this, your instance would be terminated (deleted) each time you shut down.
  • “--disable-api-termination” prevents anyone from accidentally deleting your instance. If you want to delete your instance, you must first run this command to re-enable termination:
    ec2-modify-instance-attribute --disable-api-termination false i-FFFFFFFF
    Then you can terminate your instance, delete your EBS volume, and release your elastic IP address using the command-line tools or the AWS Management Console
• Allow SSH and ICMP through firewall. This applies to all instances in the default security group:
  ec2-authorize default -P tcp -p 22 -s 0.0.0.0/0
ec2-authorize default -P icmp -t -1:-1 -s 0.0.0.0/0
  
• Your instance has a public IP address. Look it up and ping it:
  ec2-describe-instances
  
• The public IP address changes every time you stop and start your instance. Run these commands to get a static IP address and point it at your instance. Replace 1.2.3.4 with the newly allocated address. Also replace i-FFFFFFFF with your instance ID. Remember that Amazon charges for addresses that are not associated with running instances.
  ec2-allocate-address
ec2-associate-address 1.2.3.4 -i i-FFFFFFFF
  
• This would be a good time to make a DNS A-record pointing to the static IP address. A name like leonhard.restbackup.com is much easier to remember than 50.18.62.58.

Start/Stop Scripts

• Create the file Documents\aws\start-leonhard.restbackup.com.bat with the following code. Replace 1.2.3.4 and i-FFFFFFFF with the appropriate values.

  call %USERPROFILE%\Documents\aws\ec2env.bat
  call ec2-start-instances i-FFFFFFFF
  sleep 10
  call ec2-associate-address 1.2.3.4 -i i-FFFFFFFF
  pause
  

• Create the file Documents\aws\stop-leonhard.restbackup.com.bat with the following code. Replace i-FFFFFFFF with your instance ID.

  call %USERPROFILE%\Documents\aws\ec2env.bat
  call ec2-stop-instances i-FFFFFFFF
  pause
  

Now you can easily start and stop your instance. Running the start script has no effect when your instance is already running. Similarly you can run the stop script multiple times with no ill effects.

Remember that Amazon charges for your EBS volume and static IP address even when your instance is stopped.

Install FreeNX

• Run PuTTY from the Start Menu, connect to the static IP address, and log in as ubuntu. PuTTY will use the key from pageant.
• Resize the filesystem to match the EBS volume
  sudo resize2fs /dev/sda1
• Remove the ubuntu user password
  sudo passwd -d ubuntu
• Add the freenx package repository
  sudo add-apt-repository ppa:freenx-team
• Use packages built for Ubuntu 9.10 Karmic because the Ubuntu 10.04 Lucid packages have a problem
  sudo sed -i 's/lucid/karmic/' /etc/apt/sources.list.d/freenx-team-ppa-lucid.list
• Download lists of available packages
  sudo apt-get update
• Install the FreeNX server packages and dependencies
  sudo aptitude install freenx
• Configure the FreeNX server scripts and make a new client ssh key
  yes |sudo /usr/lib/nx/nxsetup --install
• Copy the client ssh key to your Windows machine
  sudo cat /var/lib/nxserver/home/.ssh/client.id_dsa.key
  Save the contents of the file to Documents\aws\freenx-client.id_dsa.key
• Enable ssh logins with passwords
  sudo sed -i 's/^PasswordAuthentication no$/PasswordAuthentication yes/' /etc/ssh/sshd_config
sudo /etc/init.d/ssh reload
• Install gnome-terminal
  sudo apt-get install xterm gnome-terminal
• Add your user account
  sudo adduser leonhard
• Give yourself sudo permissions
  sudo visudo
  Add this line to the end of the file:
  leonhard ALL=(ALL) ALL
• If you have trouble, consult the Ubuntu FreeNX Documentation

Log In

• Run “NX Client for Windows” from the Start Menu. The “NX Connection Wizard” opens.
• Enter leonhard.restbackup.com in the Session and Host boxes
• Click Next
• Change selection from KDE to Custom and click the Settings button
• Select “Run the following command” and enter /usr/bin/gnome-terminal in the box
• OK, Next, Finish.
  
• The NX Client window appears. Enter your username and password.
• Click Configure, click Key, and replace the DSA Key with the contents of the file Documents\aws\freenx-client.id_dsa.key
• Click Save, Click OK, Click Yes to save the configuration
  
• Click Login
• A gnome-terminal window should appear after about 30 seconds.

Happy developing on EC2!

In my next article, I will show you how to automatically back up your source code to RestBackup™ on commit.

About the Author

Michael Leonhard is the creator of RestBackup™, a new web service helping apps to backup & restore.

Comments

These comments were submitted when this blog was hosted at Posterous.

prmr said...

Hi Michael,

Great post. I was wondering what the responsiveness of using EC2 as your desktop is like? Could you watch a YouTube video, for example?

James.

---

Michael Leonhard said...

James> The responsiveness is great for Eclipse and other software that primarily draws text and lines. Web browsing has some lag, as the rendered pages are downloaded through the ssh tunnel. YouTube is streaming uncompressed video to the screen, so it gets about 1 fps. I just run my browser locally. YouTube works great on Windows!

--Michael

---

Tim Dysinger said...

James,

No you can't watch videos unless you are on an awesome super fast pipe connected to amazon. And AFAIK there's no sound. It's similar to VNC in desktop viewing quality.

IMO it's not a usable "desktop" for many tasks.

-Tim

---

lobo said...

who said "no sound" ? this is WRONG :)

I personally use it on a dedicated server (30 euros / month, unlimited bandwith) rather than AWS .. setup is the same tho:

http://ubuntuforums.org/showthread.php?t=1092258

NX works well with ESD (enlightment sound daemon) but since most distros nowadays are using PulseAudio, you have to do another step, outlined in that ubuntuforums.org link

happy hacking !

---

InformedPatriot said...

Great rundown. Would this work just as well with a Mac?

---

Michael Leonhard said...

InformedPatriot> It should work just as well on a Mac. You can download the NX Client for Mac at http://www.nomachine.com/download-client-macosx.php

---

Kin Lane said...

Great overview. Definitely will help people get setup.

I've been using a small instance as my primary desktop for over year...it really helped me transition from a Windows workstation to my Mac.

As well as stay machine independent. Everything I have is on my EC2 workstation and backed up to my S3.

---

chezbut said...

do you actually only pay $26/mo with active usage for your EC2 dev instance? That's cheap!

---

Chris Boyle liked this post.

---

Paul A Houle said...

Don't forget that if you like Windows, you can do the same thing with Windows and log in with RDP. Personally if I want to run GUI apps, I like Windows, not just because the GUI is better developed, but it's dead easy to log into into a Windows machine with RDP if you're running Windows, MacOS, or Linux.

Overall, AWS is awesome, even though it would be nice to have finer grained steps in the instances you can rent. Some other providers, like Softlayer, let you pick an arbitrary number of CPUs, arbitrary amount of RAM and all that, but, starting out, I find that much choice to be intimidating.

---

tommy molecule said...

fyi: your private key, not so private anymore.

---

Kevin Neely said...

I thought Amazon now offered a totally free low-resource instance on EC2. That would bring the desktop cost to very little. Unless of course, using an instance as a desktop consumes more resources than the free instance allows.

---

Jimbo said...

Why would you be this technical and setup this much...then still use windows for you primary machine..what a waste.

---

Michael Leonhard said...

Paul A Houle> Yeah, I used Windows on EC2 for the first time this week and found that it works very well. I wish there was an RDP server for Linux that had good performance like FreeNX.

tommy molecule> You can type in the private key and try it out :P

Kevin Neely> Micro instances may not have enough cpu for heavy development. There's an explanation under "Selecting Instance Types" at http://aws.amazon.com/ec2/instance-types/

Jimbo> I got used to a dual-machine setup at Amazon. There I had a Linux desktop and a Windows laptop. Having a good Linux dev machine is great. Having one colocated with your upstream dependencies is better. Also, Windows 7 rocks.

---

Andrey Smirnov said...

Great post Michael. I just learned about FreeNX :)

---

AlexFielder said...

Could you use one of these services to run a game server? (#BFBC2 or #Left4Dead2 for instance)

---

Shane said...

One of the cons you list is a lack of cheap 64 bit server type. I use the t1.micro for this desktop use scenario with 64bit. It's only 2 cents an hour, unless I start it as a Spot request (which has many disadvantages for desktop use) then it's even cheaper. Moving up to the next 64bit server type, though, is a big jump in price. The lack of RAM, but the dual core nature of the t1.micro seems to work well enough for me, though.

---

Kin Lane said...

I don't think this solution is for everyone. For me...I've been doing it for a while now and it helped me migrate my world to the mac.

I run small instance mostly....the costs are a business expense....and justified with the type of work I do. Having a public IP address on my centralized workstation helps a lot.

Plus the ability to scale up and down based upon doing some harvesting or processing is nice.

I don't experience any latency...unless the Internet connect I'm on is slow.

I also backup the AMI and all the files + data to S3 and simultaneously to the Rackspace cloud for redundancy. And cover my ass.

I think Michael took some time and laid out a great plan for people to follow. If you think its dumb....I'd move on. But I think your limiting your perspective on whats possible out there because it definitely has its purpose for some.

I think its funny how people get hung up on the costs? If you run a professional shop. Its cost of doing business. I have a central place to host all my files, projects, give places for my devs and clients to access my resources. I don't worry about losing a workstation. Keeps me machine independent.

Thanks Michael. Appreciate your work here.

---

Steve S said...

>The responsiveness is great for Eclipse

I used to use FreeNX on windows to connect to an ubuntu desktop running at my office, primarily for Eclipse. It was pretty unresponsive for me. I always thought that was because the window was drawn by the JRE or something like that. My ctrl, shift and alt keys would get stuck sometimes, or not even register. Sometimes sections of a window would blank out and I needed to hide and restore the window to redraw it. Sometimes text would blank out.

In order to improve my experience, I ended up configuring a separate development environment at home and checking out a copy of my repositories from my remote machines.

I always thought that FreeNX felt slower than it should. I know my connection to work is not in the same league as Amazon EC2, but there is more than enough bandwidth to spare for one desktop session, since I would more than likely be the only person using the connection after hours.

When I asked around on IRC, I heard that the general consensus was that this was the level of performance that I should expect, and if I wanted a smoother experience I should stick to running things locally.

Do you have any tips I could try to improve the responsiveness of Eclipse over FreeNX? Thanks.

---

mikew909 said...

great article ! the best explanation of ec2 setup i've seen so far

---

Alex Covic liked this post.

---

mikew909 said...

Just wondering anyone had an issue with NoMachine - Cannot initialize display service - everything seemed to go fine until i try to login through the client

---

Michael Leonhard said...

mikew909> NX Client gives me that error on Windows Server 2008. It works fine under Windows 7.

---

stephen said...

I believe I have followed the following steps, but then I enter the following,
bash-3.2$ ec2-run-instances --group default --key sshkeypair --block-device-mapping "/dev/sda1=:16:off" --instance-initiated-shutdown-behavior stop --disable-api-termination ami-714ba518

but get the following response,
Client.InvalidAMIID.NotFound: The AMI ID 'ami-714ba518' does not exist

Am I doing anything obviously wrong?

---

Michael Leonhard said...

stephen> You can get that error if you run the command against one region and specify an AMI that only exists in another region. Try setting EC2_URL=https://ec2.us-east-1.amazonaws.com/ and try again.

C:\>ec2-describe-images --region us-east-1 ami-714ba518
IMAGE ami-714ba518 099720109477/ebs/ubuntu-images/ubuntu-lucid-10.04-i386-server-20100427.1 099720109477 available public i386 machine aki-754aa41c ebs BLOCKDEVICEMAPPING /dev/sda1 snap-5e4bec36 15

C:\>ec2-describe-images --region us-west-1 ami-714ba518
Client.InvalidAMIID.NotFound: The AMI ID 'ami-714ba518' does not exist

---

Adam said...

I've followed all these steps, and can log into NX and I get the gnome-terminal. Am I supposed to be seeing a GUI, or is the purpose to kick off the GUI such as Eclipse or Thunderbird from the command line?

---

Michael Leonhard said...

Adam> Yes, you can start graphical programs from the command line. Add '&' to the end of your command to run it in the background.

---

Stefan said...

Thanks! Keep up the good work!

---

David said...

Thanks for the great guide. Until I read this article I didn't realize that it could be done - the key fact I was missing is the EBS device which gives your dev machine the state you need (you know, for your ~/.emacs), which you don't have for arbitrary production EC2 instances. I'm going to try this out!