Auto Scaling Strategies For Windows Azure, Amazon’s EC2 And Other Cloud Platforms

The strategies discussed in this article can be applied to any cloud platform that has an ability to dynamically provision compute resources, even though I rely on examples from AzureWatch auto scaling and monitoring service for Windows Azure

The topic of auto scaling is an extremely important one when it comes to architecting cloud-based systems.  The major premise of cloud computing is its utility based approach to on-demand provisioning and de-provisioning of resources while paying only for what has been consumed.  It only makes sense to give the matter of dynamic provisioning and auto scaling a great deal of thought when designing your system to live in the cloud.  Implementing a cloud-based application without auto scaling is like installing an air-conditioner without a thermostat: one either needs to constantly monitor and manually adjust the needed temperature or pray that outside temperature never changes.

Many cloud platforms such as Amazon’s EC2 or Windows Azure do not automatically adjust compute power dedicated to applications running on their platforms.  Instead, they rely upon various tools and services to provide dynamic auto scaling.  For applications running in Amazon cloud, auto-scaling is offered by Amazon itself via a service CloudWatch as well as third party vendors such as RightScale.  Windows Azure does not have its own auto scaling engine but third party vendors such as AzureWatch can provide auto scaling and monitoring.

Before deciding on when to scale up and down, it is important to understand when and why changes in demand occur.  In general, demand on your application can vary due to planned or unplanned events.  Thus it is important to initially divide your scaling strategies into these two broad categories: Predictable and Unpredictable demand.

The goal of this article is to describe scaling strategies that gracefully handle unplanned and planned spikes in demand.  I’ll use AzureWatch to demonstrate specific examples of how these strategies can be implemented in Windows Azure environment.  Important note: even though this article will mostly talk about scale up techniques, do not forget to think about matching scale down techniques. In some cases, it may help to think about building an auto scaling strategy in a way similar to building a thermostat.

Unpredictable demand

Conceptualizing use-cases of rarely occurring unplanned spikes in demand is rather straight forward.  Demand on your app may suddenly increase due to a number of various causes, such as:

  • an article about your website was published on a popular website
  • CEO of your company just ordered a number of complex reports before a big meeting with shareholders
  • your marketing department just ran a successful ad campaign and forgot to tell you about the possible influx of new users
  • a large overseas customer signed up overnight and started consuming a lot of resources

Whatever the case may be, having an insurance policy that deals with such unplanned spikes in demand is not just smart.  It may help save your reputation and reputation of your company.  However, gracefully handling unplanned spikes in demand can be difficult.  This is because you are reacting to events that have already happened.  There are two recommended ways of handling unplanned spikes:

Strategy 1: React to unpredictable demand

When utilization metrics are indicating high load, simply react by scaling up.  Such utilization metrics can usually include CPU utilization, amount of requests per second, number of concurrent users, amounts of bytes transferred, or amount of memory used by your application.  In AzureWatch you can configure scaling rules that aggregate such metrics over some amount of time and across all servers in the application role and then issue a scale up command when some set of averaged metrics is above a certain threshold.  In cases when multiple metrics indicate change in demand, it may also be a good idea to find a “common scaling unit”, that would unify all relevant metrics together into one number.

Strategy 2: React to rate of change in unpredictable demand

Since scale-up and scale-down events take some time to execute, it may be better to interrogate the rate of increase or decrease of demand and start scaling ahead of time: when moving averages indicate acceleration or deceleration of demand.  As an example, in AzureWatch’s rule-based scaling engine, such event can be represented by a rule that interrogates Average CPU utilization over a short period of time in contrast to CPU utilization over a longer period of time

(Fig: scale up when Average CPU utilization for the last 20 minutes is 20% higher than Average CPU utilization over the last hour and Average CPU utilization is already significant by being over 50%)

Also, it is important to keep in mind that scaling events with this approach may trigger at times when it is not really needed: high rate of increase will not always manifest itself in the actual demand that justifies scaling up.  However, in many instances it may be worth it to be on the safe side rather than on the cheap side.

Predictable demand

While reacting to changes in demand may be a decent insurance policy for websites with potential for unpredictable bursts in traffic, actually knowing when demand is going to be needed before it is really needed is the best way to handle auto scaling.  There are two very different ways to predict an increase or decrease in load on your application.  One way follows a pattern of demand based on historical performance and is usually schedule-based, while another is based on some sort of a “processing queue”.

Strategy 3: Predictable demand based on time of day

There are frequently situations when load on the application is known ahead of time.  Perhaps it is between 7am and 7pm when a line-of-business (LOB) application is accessed by employees of a company, or perhaps it is during lunch and dinner times for an application that processes restaurant orders.  Whichever it may be, the more you know at what times during the day the demand will spike, the better off your scaling strategy will be.  AzureWatch handles this by allowing to specify scheduling aspects into execution of scaling rules.

Strategy 4: Predictable demand based on amount of work left to do

While schedule-based demand predictions are great if they exist, not all applications have consistent times of day when demand changes.  If your application utilizes some sort of a job-scheduling approach where the load on the application can be determined by the amount of jobs waiting to be processed, setting up scaling rules based on such metric may work best.  Benefits of asynchronous or batch job execution where heavy-duty processing is off-loaded to back-end servers can not only provide responsiveness and scalability to your application but also the amount of waiting-to-be-processed job can serve as an important leading metric in the ability to scale with better precision.  In Windows Azure, the preferred supported job-scheduling mechanism is via Queues based on Azure Storage.  AzureWatch provides an ability to create scaling rules based on the amount of messages waiting to be processed in such a queue.  For those not using Azure Queues, AzureWatch can also read custom metrics through a special XML-based interface.

Combining strategies

In the real world, implementing a combination of more than one of the above scaling strategies may be prudent.  Application administrators likely have some known patterns for their applications behavior that would define predictable bursting scenarios, but having an insurance policy that would handle unplanned bursts of demand may be important as well.  Understanding your demand and aligning scaling rules to work together is key to successful auto scaling implementation.

By Igor Papirov /Paraleap Technologies

About CloudTweaks

Established in 2009, CloudTweaks is recognized as one of the leading authorities in connected technology information and services.

We embrace and instill thought leadership insights, relevant and timely news related stories, unbiased benchmark reporting as well as offer green/cleantech learning and consultive services around the world.

Our vision is to create awareness and to help find innovative ways to connect our planet in a positive eco-friendly manner.

In the meantime, you may connect with CloudTweaks by following and sharing our resources.

View All Articles

Sorry, comments are closed for this post.

SaaS Freemium Models and the Hidden Cost of Free

SaaS Freemium Models and the Hidden Cost of Free

SaaS Freemium Models We’ve all been lured into sexy “try before you buy” freemium models that provide just the right amount of functionality to get you started. Yet, it’s not quite enough to complete the job. “Getting the job done” often requires stepping up to a paid or premium version that provides more functionality, capabilities…

The Concept Of Securing IoT To Secure Your Building

The Concept Of Securing IoT To Secure Your Building

Securing IoT Ah, security. It is the dulcet tone of a symphony that we play over and over in the IT world. IoT (Internet of Things) and the myriad of connected devices allow us some intriguing security options. For example, in a mesh array of sensors, you could effectively force users to correctly identify themselves…

It Takes a Village to Raise a Cloud

It Takes a Village to Raise a Cloud

The Community Cloud “We are special. We have unique needs. Our demands for compliance and security go way above your average organization. There is no way we are going to put our precious data in a public cloud. It is just a non-starter.” Sound familiar? This kind of reasoning is often the driver behind either…

Encrypting Your Cloud Data For Extra Protection

Encrypting Your Cloud Data For Extra Protection

Encrypting Your Cloud Data Encrypting data is one of the best ways of protecting your data as it moves to the cloud. The only thing better than encrypting your data, is not storing your data at all. Let’s first look at the case of using file sharing applications such as Dropbox. If you are the…

5 Companies That Took On The Small Business Cloud – And Lost

5 Companies That Took On The Small Business Cloud – And Lost

5 Companies That Took On The Small Business Cloud The growth of cloud based business support systems has been staggering, and while some major software houses have quickly positioned themselves as forerunners in the move to the Cloud, there are several high-flying companies that have so far failed to provide small businesses with the level…

New Smartphones From Apple, Samsung and HTC Promise To Light Up 2016

New Smartphones From Apple, Samsung and HTC Promise To Light Up 2016

New Smartphones from Apple, Samsung and HTC (Sponsored post courtesy of Verizon Wireless) The launch of the Galaxy S7 Edge at the Mobile World Congress in Barcelona during February was the first shot in a vintage year for mobile phones. The S7 is an incredible piece of hardware, but launches from HTC and Apple later in the…

Featured Sponsored Articles
How Successful Businesses Ensure Quality Team Communication

How Successful Businesses Ensure Quality Team Communication

Quality Team Communication  (Sponsored post courtesy of Hubgets) Successful team communication and collaboration are as vital to project and overall business success as the quality of products and services an organization develops. We rely on a host of business tools to ensure appropriate customer interactions, sound product manufacturing, and smooth back-end operations. However, the interpersonal relationships…

Featured Sponsored Articles
How To Develop A Business Continuity Plan Using Internet Performance Management

How To Develop A Business Continuity Plan Using Internet Performance Management

Internet Performance Management Planning CDN Performance Series Provided By Dyn In our previous post, we laid out the problems of business continuity and Internet Performance Management in today’s online environment.  In this article, we will take a look at some of the ways you can use traffic steering capabilities to execute business continuity planning and…

Featured Sponsored Articles

CloudTweaks is recognized as one of the leading influencers in cloud computing, infosec, big data and the internet of things (IoT) information. Our goal is to continue to build our growing information portal by providing the best in-depth articles, interviews, event listings, whitepapers, infographics and much more.

Sponsor