Amazon EC2

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. Amazon EC2 reduces the time required to obtain and boot new server instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change. EC2 is effectively virtual machines in the cloud.

Amazon EC2 changes the economics of computing by allowing you to pay only for capacity that you actually use. Amazon EC2 provides developers the tools to build failure resilient applications and isolate themselves from common failure scenarios.

See the EC2 FAQs for more information.

Pricing Options

On Demand

Pay a fixed rate by the hour (or by the second) with no commitment.

Use Cases:

  • Applications with short term, spiky, or unpredictable workloads that cannot be interrupted.
  • Applications that are being developed or tested on Amazon EC2 for the first time.
  • Users that want the low cost and flexibility of Amazon EC2 without any up-front payment or long-term commitment.

Reserved

Provides you with a capacity reservation, and offers a significant discount on the hourly charge for an instance. 1 year or 3 year terms.

Use cases:

  • Applications with a steady state or predictable usage.
  • Applications that require reserved capacity.
  • Users are able to make upfront payments to reduce their total computing costs even further.

Spot

Enables you to bid whatever price you want for instance capacity, providing for even greater savings if your applications have flexible start and end times. Once the price goes below your bid, your instance will start up.

Use cases:

  • Applications that have flexible start and end times.
  • Applications that are only feasible at very low compute prices. Example usage might be big data processing that can happen at 3AM.
  • Users with urgent computing needs for large amounts of additional capacity.

Note: with spot instances, if you terminate the instance, you pay for the hour. If AWS terminates the spot instance, you get the hour it was terminated in for free.

Dedicated Hosts

Physical EC2 server dedicated for your use. This in contrast to the other types which are shared. Dedicated Hosts can help you reduce costs by allowing you to use your existing server-bound software licenses. Can be purchased on-demand (hourly). Can also be purchased as a reservation for up to 70% off the on-demand price.

Use cases:

  • Useful for regulatory requirements that may not support multi-tenant virtualization.
  • Great for licensing which does not support multi-tenancy or cloud deployments.

Instance Types

ec2-instance-types.png

These types can be remembered with…

  • D: Density.
  • R: RAM.
  • M: Main choice for general purpose applications.
  • C: Compute.
  • G: Graphics.
  • I: IOPS.
  • F: Field Programmable Gate Array.
  • T: "Tiny", cheap general purpose.
  • P: Graphics (think Pics)
  • X: Extreme memory.

dr-mcgift-px.png

Amazon Machine Image (AMI)

When first creating an EC2 instance you get to choose an Amazon Machine Image (AMI). An AMI is a virtual appliance. The main component is a read-only filesystem image that includes an operating system and any additional software. You can create AMIs from both Volumes and Snapshots.

If you set up an instance, take a snapshot, and then create an AMI, you can use that AMI to launch a new instance that has the same setup. This template can ease future set ups. Note that you cannot delete a snapshot of an EBS Volume that is used as the root device of a registered AMI.

AMIs are regional. You can only launch an AMI from the region in which it is stored. However you can copy AMIs to other regions using the console, command line, or the Amazon EC2 API.

AMI Types

You can select your AMI based on: region, operating system, architecture (32-bit or 64-bit), launch permissions, storage for the root device (root device volume).

All AMIs are categorized as either backed by Amazon EBS or backed by Instance Store. For EBS Volumes, the root device for an instance launched form the AMI is an Amazon EBS volume created from an Amazon EBS snapshot. For Instance Store Volumes, the root device for an instance launched from the AMI is an instance store volume created from a template stored in Amazon S3.

Instance Store Volumes are sometimes called Ephemeral Storage. Instance Store Volumes cannot be stopped. If the underlying host fails, you will lose your data. EBS backed instances can be stopped. You will not lose the data on this instance if it is stopped. You can reboot both, you will not lost your data. By default, both ROOT volumes will be deleted on termination, however with EBS volumes, you can tell AWS to keep the root device volume.

Termination

Terminating an instance shuts it down and deletes it. On an EBS-backed instance, the default action is for the root EBS volume to be deleted when the instance is terminated.

You can control whether an EBS root volume is deleted when its associated instance is terminated. The default delete-on-termination behaviour depends on whether the volume is a root volume, or an additional volume. By default, the DeleteOnTermination attribute for root volumes is set to 'true.' However, this attribute may be changed at launch by using either the AWS Console or the command line. For an instance that is already running, the DeleteOnTermination attribute must be changed using the CLI.

Termination Protection can be set on EC2 instances to prevent termination until the option is disabled. Termination Protection is turned off by default.

Instance Lifecycle

ec2-instance-lifecycle.png

Security Groups

A Security Group is a virtual firewall. It's a set of rules that control the traffic to and from your instance.

You can have any number of EC2 instances within a Security Group. You can also have multiple Security Groups attached to EC2 instances.

Changes to Security Groups take effect immediately.

Rules

Security Group rules are a whitelist; you can specify allow rules, but not deny rules. This means that you cannot block specific IP addresses using Security Groups (for this purpose use Network Access Control Lists).

Rules can be inbound or outbound. Rules are stateful; If you create an inbound rule allowing traffic in, that traffic is automatically allowed back out again.

Defaults

All inbound traffic is blocked by default. All outbound traffic is allowed by default.

SSH

You can ssh into an EC2 instance with

#+BEGIN_SRC sh ssh ec2-user@<public-ip> -i keyPair.pem #+END_ARC

AIM Roles and EC2

Manually setting credentials with aws configure can be a security risk on EC2 instances. If an EC2 instance is compromised, these credentials can be stolen. In addition, setting credentials in this way can be difficult to manage. Roles can improve security and ease management. Create a role using IAM. Roles can be attached to EC2 instances when they are created or even when they are running.

You can change the permissions to a role, even if that role is already assigned to an existing EC2 instance, and these changes will take effect immediately.

Bootstrap Scripts

When you launch an instance in Amazon EC2, you can choose to run user data. This may be a shell script. Specify a script under Advanced Details during the launch wizard. Use a shebang (#!) at the start of the script. You can see the contents of user data with curl http://169.254.169.254/latest/user-data/.

Metadata

Retrieve metadata about your current EC2 instance with curl http://169.254.169.254/latest/meta-data/. This will give a list of variables. To get the public IP address, run curl http://169.254.169.254/latest/meta-data/public-ipv4.

Auto Scaling

An Auto Scaling group contains a collection of EC2 instances that share similar characteristics and are treated as a logical group for the purposes of instance scaling and management. For example, if a single application operates across multiple instances, you might want to increase the number of instances in that group to improve the performance of the application, or decrease the number of instances to reduce costs when demand is low. You can use the Auto Scaling group to scale the number of instances automatically based on criteria that you specify, or maintain a fixed number of instances even if an instance becomes unhealthy.

Create an Auto Scaling group through the EC2 page on the AWS console. First you'll need to create a launch configuration. Creating this template looks very similar to a normal EC2 launch. Next, create an Auto Scaling group. Specify how many instances you want to run in the group. Your group will maintain this number of instances, and replace any that become unhealthy or impaired. You can optionally configure your group to adjust in capacity according to demand, in response to Amazon CloudWatch metrics.

Placement Group

A Placement Group is a logical grouping of instances within a single Availability Zone. Using placement groups enables applications to participate in a low-latency, 10 Gbps network. Placement groups are recommended for applications that benefit from low network latency, high network throughput, or both.

  • A placement group can't span multiple Availability Zones (make sense, we're looking for high network speed).
  • The name you specify for a placement group must be unique within your AWS account.
  • Only certain types of instances can be launched in a placement group (Compute Optimized, GPU, Memory Optimized, Storage Optimized).
  • AWS recommend homogenous instances within placement groups.
  • You can't merge placement groups.
  • You can't move an existing instance into a placement group. You can create an AMI from your existing instance, and then launch a new instance from the AMI into a placement group.

Placement Group Types

AWS has two types of Placement Groups:

  • Clustered Placement Groups
    • Always going to be in one availability zone. Used for low-latency, high throughput.
  • Spread Placement Groups
    • Guarantees separate hardware.