When AWS Takes the Wheel

Amazon Web Services offers capabilities that automate processes and functions
of cloud applicationspromising to save time and money. But first, organizations
must establish a firm understanding of their workloads and patterns.

EDITORS NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

ITS A MATTER OF
CONTENT

DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

EDITORS
NOTE

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

Up Your Apps With AWS Automation

From automated bill payments to mobile

applications that remind you to feed your cat,
automation has come a long waysaving people time and money.
It can do the same for those in the IT world.
Amazon Web Services offers automation services like CloudWatch and Auto Scaling to
minimize user responsibility and maximize
efficiency of cloud applications. In this threepart guide, we review some of the benefits of
these services. Among them, automatically
adding or removing instances, avoiding scaling
inaccuracies and producing better applications.
Software engineer Brian Tarbox kicks things
off with a look at reactive scalingwhich
enables a program to automatically react to load
changes in its environmentand how AWSs
Auto Scaling service is putting the concept to
work.
Next, technology journalist George Lawton

WHEN AWS TAKES THE WHEEL

interviews businesses about their experiences

with AWS CloudFront. Its designed to improve
dynamic content and optimize delivery of
static content quickly and cost-effectively. It
does a better job than Amazon Simple Storage
Services, Lawton finds, but he warns, It is not
a silver bullet for every source of delay in Web
performance.
To close, Tarbox delves deeper into public
cloud scaling and reviews the pros and cons
of using predictive versus automated scaling.
Auto scaling workloads according to demand is
great for business, but there are other options
to consider. Choosing the right one for your
AWS cloud, however, depends on how much
you know about your workloads and their patterns, Tarbox said. n
Katherine Wiley
Site Editor, SearchAWS

PROGRAMMING

AWS Gets Reactive With Auto Scaling

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

Reactive programming, which enables

a program to automatically react to load
changes in its environment and continue running, is one of many IT buzzwords thrown
around lately. But the concept isnt new. Many
programs that are labeled reactive were always
supposed to be resilient in the face of change.
Amazon Web Services offers an Auto Scaling
service that allows the system to react to such
changes.
Reactivity is really a systems concept. Each
program should be resilient to change, and the
overall system should add, remove and modify
programs and resources as needed.
Auto Scaling enables a system to add resources, typicallyAWS Elastic Compute
Cloud (EC2) instances, when the load increasesand remove them when the load diminishes. There are four types of Auto Scaling:
fixed, manual, scheduled and dynamic.
Fixed Auto Scaling ensures that AWS

WHEN AWS TAKES THE WHEEL

replaces instances that fail, while manual Auto

Scaling lets you add or remove instances by
hand. Scheduled Auto Scaling is used when
your load varies only by date or time, and
dynamic Auto Scaling is for when your load
changes unpredictably.
You may think the term load is predefined
to mean something like CPU percentage or
Ubuntu load average or Web-response time,
but cloud administrators have full control over
what constitutes load. You can use CloudWatch
metrics to define these conditions and then
group them into one or more policies, which
can be predefined metrics or ones you create
for specific system needs.

LOADED DEMANDS

To understand AWS Auto Scaling, you need to

understand two important concepts: hysteresis
and EC2 pricing.

PROGRAMMING

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

Hysteresis means adding a lag time to the

response a system has to an event. Take your
home thermostat, for example. If you set it to
68 degrees in the winter, the furnace wont
turn on until the temperature is slightly below
68, and then it will run until it reaches slightly
above 68. Thats so the furnace wont run for a
minute, shut off and then turn back on again.

To understand Amazon Web

Services Auto Scaling, you
need to understand two important concepts: hysteresis and
EC2 pricing.
With EC2 pricing, AWS charges for instances
by the hour. If you start an instance and then
kill it after 10 minutes, youre charged for a full
hour. If, within a single hour, you terminate and
start a new instance five times, you are charged
for five hours.
Combine these concepts with Auto Scaling
and it makes sense: You want the system
to react to changes in load. If that load is
highly dynamic, however, you should moderate

WHEN AWS TAKES THE WHEEL

your response to it. This refers more often to

removing instances than to adding them. When
your load increases, you often have no option
but to add capacity by adding instances. When
the load decreases, you have some flexibility
in deciding when to remove instances. If an
instance has been running for only a fraction of
the current hour, it makes sense to leave it running for a while in case the load picks up again.
You can configure AWS CloudWatch alarms
to fire only after a condition has lasted for a
specified amount of time. For example, you
could create an alarm that triggers the removal
of instances after some load metric dips below
a threshold for more than 30 minutes. That
way, instances wont be removed during brief
dips in load only to have to be re-created
within the same charging hour.

SPECIFIED INSTANCES

An Auto Scaling Policy determines the speed

at which instances ramp up and down. You can
have policies that add or remove one instance
at a time and others that add or remove a set
percentage of instances all at once.

PROGRAMMING

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING

The Auto Scaling group specifies the availability zone of the instances as well as the
minimum, maximum and desired number of
instances. You can also think of this as your
safety net; no matter how badly you misconfigure things, you cant accidently create more
instances than the group maximum.
In the Auto Scaling launch configuration,
you describe what type of instances you want,
specifying, among other things, the Amazon

WITH AWS CLOUDFRONT,

ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL

Machine Image and instance type. There are

several options for creating instances,and you
can specify any or all of these in the launch
configuration.
Overall, the process would require you to put
data into a metric (if using a custom metric)
and then trigger an alarm. The alarm will trigger a policy that creates or removes instances
according to the configuration within the
group. Brian Tarbox

DELIVERY

With AWS CloudFront, Its a Matter of Content

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

The primary use case for Amazon Web

Services CloudFront lies in optimizing the
delivery of static content to users. But Amazon
has raised the bar for the content distribution
network with features for accelerating dynamic
content as well. This makes the service more
useful for non-video-related applications, such
as speeding the distribution of user-generated and personalized content. Also, Amazon
charges the same for distributing dynamic content as it does for distributing static content.
These costs start on par with AWSs Simple
Storage Service (S3) but can drop as volume
grows.
For example, TwitPic was one of the early
adopters of CloudFront for improving the performance of its image-sharing application. The
service allows users to post images to Twitter
and other social media platforms. We used
CloudFront to deliver user-uploaded images
to our 50 million registered users. We served

WHEN AWS TAKES THE WHEEL

over 50 billion monthly API calls using CloudFront, said Steve Corona, formerly the CTO of
TwitPic and currently a principal architect at
Bigcommerce.
Leveraging CloudFront allowed TwitPic to
scale without having to build a large IT department. The company would have otherwise
faced a much greater challenge in growing costeffectively. Corona explained, We were delivering petabytes of traffic per month. Originally
we did this using Amazon S3 because, well,
we didnt know what we were doing. It turned
out CloudFront was actually cheaper for the
amount of traffic that we were serving. To
top that, it was much faster than serving data
straight out of S3.

UNDER THE HOOD

A typical website includes a mix of static and

dynamic content. Static content can include

DELIVERY

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

images and style sheets that are the same

across all users and are best cached at the edges
of the content distribution network (CDN).
Dynamic content includes information that
changes frequently or is personalized based on
user preferences, behavior, location or other
factors. Consequently, even so-called dynamic
content can be cached for varying periods.
Dynamically generating parts of a website
causes an increase in network traffic latency,
multiple connections to the origin server and
increased CPU load. To address this situation,
CloudFront uses a framework for processing
query string parameters and cookies for delivering the entire website without having to separate out static and dynamic content or manage
multiple domains.
CloudFront also allows application developers to configure time-to-live (TTL) values for
each file by setting the cache control headers
on files saved on the origin server. CloudFront
uses these to determine how frequently the
edge location retrieves an updated file from the
origin server. When the files change frequently,
the developer needs to set a short TTL, which
can be as little as zero seconds.

WHEN AWS TAKES THE WHEEL

CACHE BEHAVIOR BASICS

It is prudent to look at the configurations for

the cache settings used to ensure that users
get fresh data. CloudFront uses a variant of
the Least Recently Used, or LRU, caching algorithm. After content has been replicated in
the CloudFront cache, users can pull data from
the cache rather than directly from the origin
server. If the TTL setting for the content is
too short, users will effectively pull content
directly from the origin server. On the other
hand, if the TTL is set for a longer duration,
its important to ensure that the application is
configured to send an invalidation request.
For example, the Star Media Group, a division of Toronto Star Newspapers, uses CloudFront to deliver new online content of the
Toronto Star, Canadas largest daily newspaper.
With more than 3.3 million monthly active
users, scale and performance are two important
parts of the design. At the same time, different
types of content need to be set to expire with
different frequencies.
To address this challenge, the Star Media
IT team set up 19 different cache behaviors
for different content types. It also established

DELIVERY

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

design guidelines to limit the number of query

strings and to use client-side cookies rather
than server-side cookies. This makes the content more cacheable while reducing the number
of variations of content to cache.
This approach also makes it possible to
cache what is traditionally considered dynamic
content. For example, popular search results
are cached for two minutes, which increases
the speed of delivery for popular searches and
reduces the load on the origin server by eliminating the need to respond to popular queries
each time. This approach balanced caching
with the need to keep search results current
and ultimately led to a 50% improvement in
response time.

KEEPING IT FRESH

Like election results, sports scores and stock

quotes, data can be cached for short periods,
ranging from a second to a minute, to get the
benefit of caching without having to send out
stale data, said Nihar Bihani, product manager
for Amazon CloudFront. For example, in 2012,
National Public Radio delivered the election

WHEN AWS TAKES THE WHEEL

results using CloudFront and a TTL setting of

30 seconds.
In other cases, the content can be geographically generated, which makes it easier to cache
content for short periods in a given region. For
example, Earth Networks, which has developed

CloudFront can cache data

such as election results and
sports scores for short periods
to get the benefit of caching
without having to send out
stale data, Amazon says.
the popular Weather Bug application, uses
CloudFront to cache weather data. Since users
in a geographic area tend to hit the same edge
server, this makes it possible for them to pull
in weather updates without having to go back
to the origin server for each request.
Bihani said this kind of personalization based
on location is possible using specially crafted
cookies and query strings, in which data about
the user is recorded and becomes part of the
cache key. He noted, It is dynamic in the

DELIVERY

sense that it is personalized for each user, but

at the same time the data can be cached, and
so everyone in San Francisco can get a cached
copy rather than going to the Weather Bug origin server.
HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

THE VALUE OF NO CACHE

The main benefits from a CDN lie in replicating content closer to the users. But even when
the TTL is set at zero, website performance can
still improve if CloudFront is used as a proxy
for improving the connection to the origin
server rather than as a cache, Bihani said. In
these instances, the edge servers can leverage
a single, optimized connection with the backend server that can share a single Transmission
Control Protocol (TCP) connection across multiple users and reduce the number of hops that
a packet must take to the origin server.
When the first user requests content through
the edge, there is some overhead and delay
in setting up the connection with the origin
server, but subsequent requests through the
edge server can use the TCP connection that

WHEN AWS TAKES THE WHEEL

has already been established between the origin server and CloudFront edge servers, thus
reducing delays in retrieving content.
Bihani explained, The further away from
the origin server, the more packet drops that
happen on the Internet. Every time a packet
drops, we have to retransmit. With CloudFront,
not only does it use the persistent connection,
but those are also optimized for performance.
On a long connectionsuch as a user in
Singapore retrieving content from an origin
server in Virginiausing CloudFront as a
proxy can cut latency in half. A user in New
York would see some improvement, but not
as much.
While CloudFront can play a role in improving the performance of dynamic content, it is
not a silver bullet for every source of delay in
Web performance. As it stands now, the focus
of the CloudFront service has been on optimizing the network path for moving content. It
does not currently support other techniques,
such as front-end optimization, JavaScript and
CSS minification or image resizing for improving Web performance. George Lawton

MANAGEMENT

Deciding on a Cloud Scaling Method?

Do Your Homework

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

Is Amazon Web Services Auto Scaling the

best method for your enterprise workloads, or
is predictive scaling better?
The ability to scale workloads according to
demand is one of the biggest benefits of the
public cloud. And there are three public cloud
scaling methods most enterprises use: reactive,
predictive and analytic.
Choosing the right one for your AWS cloud,
however, depends on how much you know
about your workloads and their patterns.
Auto Scalingis a reactive service that adds new servers in response
to increased load on the system. A lagging
indicator is a time delay between the new load
being placed on the system and the systems
response. Depending on how long it takes for
new resources such as Elastic Compute Cloud
(EC2) instances to come online, the lag can be
measured in minutes or tens of minutes.
n

10

Reactive scaling. AWS

WHEN AWS TAKES THE WHEEL

Lag time depends on a number of factors. If

you create on-demand instances, they spin up
within tens of seconds. If, on the other hand,
you are usingspot instances, it takes on average five minutes to start. Once your instance
is running, it still may not be ready to perform
a taskdepending on whether you are using
cooked Amazon Machine Images (AMIs) or not.
A cooked instance strategy means your AMI
includes all the tools you need, so as soon as
the machine starts, you can start working.
Non-cooked images are bare-bones and require
scripts such as Puppet or Chef to download and
install packages before the system is available.
companies take a
different approach. Netflix, for example, uses
predictive scaling to anticipate load and preemptively launch additional instances. This
approach is based on the notion that load
patterns are often repeatable and therefore
n

Predictive scaling. Some

MANAGEMENT

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD
SCALING METHOD?
DO YOUR HOMEWORK

11

predictable. For example, you dont have to

go too far out on a limb to predict that Friday
night will see a higher load on the Netflixservers than Friday afternoon will.
Using this approach, Netflix provisions extra
resources ahead of the anticipated additional
load. It may incur a small increase in cost,
but it also may avoid a temporary degradation in user experience as the system avoids a
slowdown.
systems generate their
own load rather than have it created externally.
A streaming video system, for example, might
download new movies that need to be transcoded. A financial system might download new
stock data every night. In both cases, the load
on the system is a known value because the
system itself creates the load.
In these examples, the system can add or
remove instances based on its own understanding of the load.
Combining all of these cloud scaling methods
may be the best approach. If you use AWS Auto
Scaling, the component doing the scaling is the
AWS system. You dont have to worry about
n

Analytic scaling. Some

WHEN AWS TAKES THE WHEEL

monitoring it, because Auto Scaling is part of

the AWS platform. But if your system does its
own scaling, then the component that owns it
has to be made resilient in case your auto-scaling service dies.

Choosing the right public cloud

scaling method for your AWS
cloud depends on how much
you know about your workloads
and their patterns.
For this reasonand because your predictive or analytic scaling algorithms might have
errorsmany systems use AWS Auto Scaling
to establish a minimum number of instances.
Set up an Auto Scaling group with a rule such
as Keep at least one EC2 instance running and
let your custom scaling rules handle the upper
limits. The asymmetry of the AWS Auto Scaling rule mirrors the asymmetry of having too
many instances (extra cost but good response
time) versus having too few instances (lower
cost but potentially poor response time).

Brian Tarbox

ABOUT
THE
AUTHORS

BRIAN TARBOX has been doing programming since he

created a timing-and-scoring program for the Head of

the Connecticut Regatta in 1981.Though primarily an
Amazon Java programmer, he is a firm believer that engineers should be multilingual and use the best language
for the problem. Email him at briantarbox@gmail.com.

When AWS Takes the Wheel

is a SearchAWS.com e-publication.

HOME
EDITORS NOTE
AWS GETS REACTIVE
WITH AUTO SCALING
WITH AWS CLOUDFRONT,
ITS A MATTER OF CONTENT
DECIDING ON A CLOUD

GEORGE LAWTON is

a technology journalist who has

worked in systems engineering and database and software
development for the Biosphere II project in Oracle, Ariz.,
and elsewhere. Over the last 20 years, he has written
about topics such as software development, DevOps, systems integration, Java and cloud computing. Email him
at glawton@glawton.com.

Scot Petersen | Editorial Director

Jason Sparapani | Managing Editor, E-Publications
Joe Hebert | Associate Managing Editor, E-Publications
Jan Stafford | Executive Editor
Brein Matturro | Managing Editor
Katherine Wiley | Site Editor
Linda Koury | Director of Online Design
Neva Maniscalco | Graphic Designer

SCALING METHOD?
DO YOUR HOMEWORK

Rebecca Kitchens | Publisher | rkitchens@techtarget.com

Amalie Keerl | Director of Product Management
akeerl@techtarget.com
TechTarget
275 Grove Street, Newton, MA 02466
www.techtarget.com
2014 TechTarget Inc. No part of this publication may be transmitted or reproduced in any form or by any means without written permission from the
publisher. TechTarget reprints are available through The YGS Group.

STAY CONNECTED!
Follow @SearchAWS today

About TechTarget: TechTarget publishes media for information technology

professionals. More than 100 focused websites enable quick access to a deep
store of news, advice and analysis about the technologies, products and processes crucial to your job. Our live and virtual events give you direct access to
independent expert commentary and advice. At IT Knowledge Exchange, our
social community, you can get advice and share solutions with peers and experts.
COVER ART: THINKSTOCK

12

WHEN AWS TAKES THE WHEEL