The Lighter Side Of The Cloud – Big Broadband
The Lighter Side Of The Cloud – New Apps
The Lighter Side Of The Cloud – What Next?

Auto Scaling Strategies For Windows Azure, Amazon's EC2 And Other Cloud Platforms

Auto scaling strategies for Windows Azure, Amazon’s EC2 and other cloud platforms

The strategies discussed in this article can be applied to any cloud platform that has an ability to dynamically provision compute resources, even though I rely on examples from AzureWatch auto scaling and monitoring service for Windows Azure

The topic of auto scaling is an extremely important one when it comes to architecting cloud-based systems.  The major premise of cloud computing is its utility based approach to on-demand provisioning and de-provisioning of resources while paying only for what has been consumed.  It only makes sense to give the matter of dynamic provisioning and auto scaling a great deal of thought when designing your system to live in the cloud.  Implementing a cloud-based application without auto scaling is like installing an air-conditioner without a thermostat: one either needs to constantly monitor and manually adjust the needed temperature or pray that outside temperature never changes.

Many cloud platforms such as Amazon’s EC2 or Windows Azure do not automatically adjust compute power dedicated to applications running on their platforms.  Instead, they rely upon various tools and services to provide dynamic auto scaling.  For applications running in Amazon cloud, auto-scaling is offered by Amazon itself via a service CloudWatch as well as third party vendors such as RightScale.  Windows Azure does not have its own auto scaling engine but third party vendors such as AzureWatch can provide auto scaling and monitoring.

Before deciding on when to scale up and down, it is important to understand when and why changes in demand occur.  In general, demand on your application can vary due to planned or unplanned events.  Thus it is important to initially divide your scaling strategies into these two broad categories: Predictable and Unpredictable demand.

The goal of this article is to describe scaling strategies that gracefully handle unplanned and planned spikes in demand.  I’ll use AzureWatch to demonstrate specific examples of how these strategies can be implemented in Windows Azure environment.  Important note: even though this article will mostly talk about scale up techniques, do not forget to think about matching scale down techniques. In some cases, it may help to think about building an auto scaling strategy in a way similar to building a thermostat.

Unpredictable demand

Conceptualizing use-cases of rarely occurring unplanned spikes in demand is rather straight forward.  Demand on your app may suddenly increase due to a number of various causes, such as:

  • an article about your website was published on a popular website (the Slashdot effect)
  • CEO of your company just ordered a number of complex reports before a big meeting with shareholders
  • your marketing department just ran a successful ad campaign and forgot to tell you about the possible influx of new users
  • a large overseas customer signed up overnight and started consuming a lot of resources

Whatever the case may be, having an insurance policy that deals with such unplanned spikes in demand is not just smart.  It may help save your reputation and reputation of your company.  However, gracefully handling unplanned spikes in demand can be difficult.  This is because you are reacting to events that have already happened.  There are two recommended ways of handling unplanned spikes:

Strategy 1: React to unpredictable demand

When utilization metrics are indicating high load, simply react by scaling up.  Such utilization metrics can usually include CPU utilization, amount of requests per second, number of concurrent users, amounts of bytes transferred, or amount of memory used by your application.  In AzureWatch you can configure scaling rules that aggregate such metrics over some amount of time and across all servers in the application role and then issue a scale up command when some set of averaged metrics is above a certain threshold.  In cases when multiple metrics indicate change in demand, it may also be a good idea to find a “common scaling unit”, that would unify all relevant metrics together into one number.

Strategy 2: React to rate of change in unpredictable demand

Since scale-up and scale-down events take some time to execute, it may be better to interrogate the rate of increase or decrease of demand and start scaling ahead of time: when moving averages indicate acceleration or deceleration of demand.  As an example, in AzureWatch’s rule-based scaling engine, such event can be represented by a rule that interrogates Average CPU utilization over a short period of time in contrast to CPU utilization over a longer period of time

(Fig: scale up when Average CPU utilization for the last 20 minutes is 20% higher than Average CPU utilization over the last hour and Average CPU utilization is already significant by being over 50%)

Also, it is important to keep in mind that scaling events with this approach may trigger at times when it is not really needed: high rate of increase will not always manifest itself in the actual demand that justifies scaling up.  However, in many instances it may be worth it to be on the safe side rather than on the cheap side.

 

Predictable demand

While reacting to changes in demand may be a decent insurance policy for websites with potential for unpredictable bursts in traffic, actually knowing when demand is going to be needed before it is really needed is the best way to handle auto scaling.  There are two very different ways to predict an increase or decrease in load on your application.  One way follows a pattern of demand based on historical performance and is usually schedule-based, while another is based on some sort of a “processing queue”.

Strategy 3: Predictable demand based on time of day

There are frequently situations when load on the application is known ahead of time.  Perhaps it is between 7am and 7pm when a line-of-business (LOB) application is accessed by employees of a company, or perhaps it is during lunch and dinner times for an application that processes restaurant orders.  Whichever it may be, the more you know at what times during the day the demand will spike, the better off your scaling strategy will be.  AzureWatch handles this by allowing to specify scheduling aspects into execution of scaling rules.

 

Strategy 4: Predictable demand based on amount of work left to do

While schedule-based demand predictions are great if they exist, not all applications have consistent times of day when demand changes.  If your application utilizes some sort of a job-scheduling approach where the load on the application can be determined by the amount of jobs waiting to be processed, setting up scaling rules based on such metric may work best.  Benefits of asynchronous or batch job execution where heavy-duty processing is off-loaded to back-end servers can not only provide responsiveness and scalability to your application but also the amount of waiting-to-be-processed job can serve as an important leading metric in the ability to scale with better precision.  In Windows Azure, the preferred supported job-scheduling mechanism is via Queues based on Azure Storage.  AzureWatch provides an ability to create scaling rules based on the amount of messages waiting to be processed in such a queue.  For those not using Azure Queues, AzureWatch can also read custom metrics through a special XML-based interface.

Combining strategies

In the real world, implementing a combination of more than one of the above scaling strategies may be prudent.  Application administrators likely have some known patterns for their applications behavior that would define predictable bursting scenarios, but having an insurance policy that would handle unplanned bursts of demand may be important as well.  Understanding your demand and aligning scaling rules to work together is key to successful auto scaling implementation.

Contribution By: Igor Papirov /Paraleap Technologies

Follow Us!

CloudTweaks

Established in 2009, CloudTweaks.com is recognized as one of the leading authorities in cloud computing information. Most of the excellent CloudTweaks articles are provided by our own paid writers, with a small percentage provided by guest authors from around the globe, including CEOs, CIOs, Technology bloggers and Cloud enthusiasts. Our goal is to continue to build a growing community offering the best in-depth articles, interviews, event listings, whitepapers, infographics and much more...
Follow Us!

Sorry, comments are closed for this post.

Recent

Surprising Facts and Stats About Your Online Security

Surprising Facts and Stats About Your Online Security

Surprising Facts and Stats – Online Security It’s easy to get lulled into a false sense of security when browsing the web. As more and more devices join the internet of things, the risk of becoming a victim of a criminal cyber gang is increasing – we have more unsecured access points and offer would-be…

World Backup Day: Understand The Data You Are Protecting

World Backup Day: Understand The Data You Are Protecting

World Backup Day: Understand The Data You Are Protecting Did you know that 113 phones are lost or stolen every minute? What about the fact that 1 in 10 computers are infected with a virus every month? Thanks to World Backup Day, an independent initiative that was started in 2011, awareness is being raised about…

The E-Learning Market – Cloud Computing Adoption

The E-Learning Market – Cloud Computing Adoption

The E-Learning Market  We’ve talked a fair bit about e-Learning and MOOCs (Massive Open Online Courses) technologies here on CloudTweaks over the past number of years. The industry is expected to continue to grow at a brisk pace as more and more firms and educational institutions start to adopt cloud based services.  Docebo has an excellent…

Popular

The Cloud Above Our Home

The Cloud Above Our Home

Our Home – Moving All Things Into The Cloud The promise of a smart home had excited the imagination of the movie makers long ago. If you have seen any TV shows in the nineties or before, the interpretation presented itself to us as a computerized personal assistant or a robot housekeeper. It was smart,…

4 Different Types of Attacks – Understanding the “Insider Threat”

4 Different Types of Attacks – Understanding the “Insider Threat”

Understanding the “Insider Threat”  The revelations that last month’s Sony hack was likely caused by a disgruntled former employee have put a renewed spotlight on the insider threat. The insider threat first received attention after Edward Snowden began to release all sorts of confidential information regarding national security. While many called him a hero, what…

Unusual Clandestine Cloud Data Centre Service Locations

Unusual Clandestine Cloud Data Centre Service Locations

Unusual Clandestine Cloud Data Centre Service Locations Everyone knows what the cloud is, but does everybody know where the cloud is? We try to answer that as we look at some of the most unusual data centre locations in the world. Under the Eyes of a Deity Deep beneath the famous Uspenski Cathedral in the…

Cloud Migration – 10 ‘Do it Right’ Tips

Cloud Migration – 10 ‘Do it Right’ Tips

Cloud Migration – 10 ‘Do it Right’ Tips Businesses continue to adopt the cloud at break neck speed. Inherent benefits like lower operational costs, no infrastructure overheads, and quick access to better technology make cloud a very attractive proposition for businesses, especially start-ups and SMEs. However moving from legacy to the cloud environment has its…

Three Factors for Choosing Your Long-term Cloud Strategy

Three Factors for Choosing Your Long-term Cloud Strategy

Choosing Your Long-term Cloud Strategy A few weeks ago I visited the global headquarters of a large multi-national company to discuss cloud strategy with the CIO. I arrived 30 minutes early and took a tour of the area where the marketing team showcased their award winning brands. I was impressed by the digital marketing strategy…

Sponsored Posts

Skin Based Technology – The Intelligent Tattoo

Skin Based Technology – The Intelligent Tattoo

The Intelligent Tattoo I’ve got you under my skin: the rise of the intelligent tattoo As the Internet of Everything expands its reach to embrace many new areas of life, the race continues to develop more robust and accessible tools. Wearables are being whittled down from clunky watches to smart clothes and wrist bands, but…

Cloud Logo Sponsors

hp Logo CityCloud-PoweredByOpenstack-Bluesquare_logo_100x100-01
cisco_logo_100x100 vmware citrix100
Site 24x7


Contributor Spotlight

Established in 2009, CloudTweaks is recognized as one of the leading influencers in cloud computing, big data and internet of things (IoT) information. Our goal is to continue to build our growing information portal, by providing the best in-depth articles, interviews, event listings, whitepapers, infographics and much more.

Branded Content Programs

Advertising