A Closer Look At The Scalability Of Windows Azure

A Closer Look At The Scalability Of Windows Azure

The Scalability Of Windows Azure

Cloud-computing providers in general and Windows Azure in particular offer nearly infinite scalability, virtually unlimited capacity, blazing performance and extremely quick provision times. However, to properly take advantage of these great benefits, teams need to plan ahead and understand all potential pitfalls and challenges. One of the more significant differences between development of on-premise applications and cloud applications is a rather direct correlation between choices made during construction of an application and its support costs after deployment. Because of the Windows Azure pricing model, every inefficient architectural decision and every inefficient line of code will show up as an extra line item on your Azure invoice.

This article will focus on a few actionable items that you can do today to minimize the cost of your Windows Azure application tomorrow. The list of items is by no means exhaustive, but it will get you and your team thinking about the impact of your choices in a new dimension.

First, let’s analyze the popular moving blocks when it comes to Windows Azure pricing. While seemingly straightforward individually, it is their combination together – and with obvious influence of already existing non-functional requirements for scalability, performance, security, availability, etc. – that make architecture in the cloud be a complex jigsaw puzzle.

  • Compute hours – quantity and size of nodes (servers) and charged by hour when online
  • Transfer costs – data that crosses Microsoft’s data center boundaries is subject to transfer charges.
  • Azure Table Storage (ATS) costs – charge by gigabyte per month for the amount of space used
  • ATS transaction costs – charges for the amount of requests to ATS your application will make
  • Size of the SQL Azure databases – every database that you host in SQL Azure is charged by size

There are costs for other less frequently used services like Azure AppFabric or Content Delivery Network (CDN) that are not covered in this article.

Tip 1 – Avoid crossing data center boundaries

This is fairly straightforward. Data that does not leave Microsoft data center is not subject to Transfer charges. Keep your communication between compute nodes, SQL Azure, and Table Storage within the same data center as much as possible. This is especially important for applications distributed among multiple geo-locations. If you must communicate between different geo-locations, limit communication to non-transactional, batch calls that occur less frequently while utilizing compression where it makes sense to cut down on the amount of data transferred. Employ caching technologies where possible.

Tip 2 – Minimize the number of compute hours by using auto scaling

Compute hours will likely make up the largest part of your Azure bill and thus need to receive the greatest amount of attention. It is important to remember that Windows Azure does not automatically scale down the number of compute nodes, even if there is little or no demand on your application. Architect for and plan to have an automatic scaling strategy in place, where the amount of nodes increases when demand spikes up and decreases when demand tapers off. This can easily cut your bill for compute hours in half. Implementing a comprehensive auto-scaling engine can be more complex than it sounds. While there are a number of open-source examples that show the basics of how this can be done, it is also a perfect opportunity to outsource the auto-scaling to third party services such as AzureWatch.

In order for auto-scaling to be most effective, group your system components by their scaling strategies into Azure Roles. It is important to keep in mind that if you need high availability of your components and want to take advantage of Azure SLA, you will need to maintain at least two online nodes for each Azure Role you have deployed.

Tip 3 – Use both Azure Table Storage (ATS) and SQL Azure

Try to not limit yourself to a choice between ATS or SQL Azure. Instead, it would be best to figure out when to use both together to your advantage. This is likely to be one of the tougher decisions that architects will need to make, as there are many compromises between relational storage of SQL Azure and highly scalable storage of ATS. Neither technology is perfect for every situation.

On one hand accessing SQL Azure from within the boundaries of a data center is free and SQL Azure offers a familiar relational model which most developers will be comfortable with, transactions that assure data integrity, integration with popular ORM frameworks such as Entity Framework or NHibernate, and compatibility with numerous tools that work with relational databases. On the other hand, ATS offers vastly greater scalability than SQL Azure and can hold a nearly infinite amount of data at a fraction of SQL Azure’s cost. You are charged, however, for every request made to ATS, even within the boundaries of a data center.

From a cost perspective, SQL Azure makes sense when access to data is not required to be highly scalable and when the amount of data is limited. ATS makes sense for large amounts of data or when serious scalability is needed.

Tip 4 – ATS table modeling

If you have made the choice to use Azure Table Storage, you have essentially committed to converting parts of your data access components into mini database servers. Setting Blob storage aside which is primarily used for media files or documents, ATS provides three levels of data hierarchy (Table, PartitionKey, and RowKey) that can be accessed and navigated extremely efficiently. However, anything beyond that will require custom code and CPU cycles of your compute nodes. This is the key difference to work around. It would be prudent to spend a significant amount of time modeling table storage with appropriate Blobs, Tables, PartitionKeys and RowKeys to accommodate for efficient data storage and retrieval strategies. This will not only speed up your transactions and minimize the amount of data transferred in and out of ATS, but also reduce the burden on you compute nodes that will be required to manipulate data and directly translate into cost savings across the board.

Tip 5 – Data purging in ATS

Because you are charged for every gigabyte stored in ATS, it may be prudent to have a data purging strategy. However, while it may seem like a straightforward problem in a world of relational databases, this is not the case with ATS. Since ATS is not relational, deletion of each and every row from an ATS table requires two transactions. In certain cases it may be possible to delete a single row using only one transaction. Either way, this is extremely slow, inefficient and expensive. A better way would be to partition a single table into multiple versions (e.g. Sales2010, Sales2011, Sales2012, etc.) and purge obsolete data by deleting a version of a table at a time.

Conclusion

Shift to cloud computing represents a major leap forward and enables almost everyone, from small companies to large enterprise, reduce their capital expenses, minimize time to market and significantly decrease support costs. By investing even small effort into planning ahead, cloud computing can result in meaningful savings and benefits.

By Igor Papirov / Paraleap Technologies

Cloud-computing providers in general and Windows Azure in particular offer nearly infinite scalability, virtually unlimited capacity, blazing performance and extremely quick provision times.  However, to properly take advantage of these great benefits, teams need to plan ahead and understand all potential pitfalls and challenges.  One of the more significant differences between development of on-premise applications and cloud applications is a rather direct correlation between choices made during construction of an application and its support costs after deployment.  Because of the Windows Azure pricing model, every inefficient architectural decision and every inefficient line of code will show up as an extra line item on your Azure invoice.

This article will focus on a few actionable items that you can do today to minimize the cost of your Windows Azure application tomorrow.  The list of items is by no means exhaustive, but it will get you and your team thinking about the impact of your choices in a new dimension.

First, let’s analyze the popular moving blocks when it comes to Windows Azure pricing. While seemingly straightforward individually, it is their combination together – and with obvious influence of already existing non-functional requirements for scalability, performance, security, availability, etc. – that make architecture in the cloud be a complex jigsaw puzzle.

Compute hours – quantity and size of nodes (servers) and charged by hour when onlineTransfer costs – data that crosses Microsoft’s data center boundaries is subject to transfer charges.Azure Table Storage (ATS) costs – charge by gigabyte per month for the amount of space usedATS transaction costs – charges for the amount of requests to ATS your application will makeSize of the SQL Azure databases – every database that you host in SQL Azure is charged by sizeThere are costs for other less frequently used services like Azure AppFabric or Content Delivery Network (CDN) that are not covered in this article.

Tip 1 – Avoid crossing data center boundaries

This is fairly straightforward.  Data that does not leave Microsoft data center is not subject to Transfer charges.  Keep your communication between compute nodes, SQL Azure, and Table Storage within the same data center as much as possible.  This is especially important for applications distributed among multiple geo-locations.  If you must communicate between different geo-locations, limit communication to non-transactional, batch calls that occur less frequently while utilizing compression where it makes sense to cut down on the amount of data transferred.  Employ caching technologies where possible.

Tip 2 – Minimize the number of compute hours by using auto scaling

Compute hours will likely make up the largest part of your Azure bill and thus need to receive the greatest amount of attention.  It is important to remember that Windows Azure does not automatically scale down the number of compute nodes, even if there is little or no demand on your application.  Architect for and plan to have an automatic scaling strategy in place, where the amount of nodes increases when demand spikes up and decreases when demand tapers off.  This can easily cut your bill for compute hours in half.  Implementing a comprehensive auto-scaling engine can be more complex than it sounds.  While there are a number of open-source examples that show the basics of how this can be done, it is also a perfect opportunity to outsource the auto-scaling to third party services such as AzureWatch.

In order for auto-scaling to be most effective, group your system components by their scaling strategies into Azure Roles.  It is important to keep in mind that if you need high availability of your components and want to take advantage of Azure SLA, you will need to maintain at least two online nodes for each Azure Role you have deployed.

Tip 3 – Use both Azure Table Storage (ATS) and SQL Azure

Try to not limit yourself to a choice between ATS or SQL Azure.  Instead, it would be best to figure out when to use both together to your advantage.  This is likely to be one of the tougher decisions that architects will need to make, as there are many compromises between relational storage of SQL Azure and highly scalable storage of ATS.  Neither technology is perfect for every situation.

On one hand accessing SQL Azure from within the boundaries of a data center is free and SQL Azure offers a familiar relational model which most developers will be comfortable with, transactions that assure data integrity, integration with popular ORM frameworks such as Entity Framework or NHibernate, and compatibility with numerous tools that work with relational databases.  On the other hand, ATS offers vastly greater scalability than SQL Azure and can hold a nearly infinite amount of data at a fraction of SQL Azure’s cost.  You are charged, however, for every request made to ATS, even within the boundaries of a data center.

From a cost perspective, SQL Azure makes sense when access to data is not required to be highly scalable and when the amount of data is limited.  ATS makes sense for large amounts of data or when serious scalability is needed.

Tip 4 – ATS table modeling

If you have made the choice to use Azure Table Storage, you have essentially committed to converting parts of your data access components into mini database servers.  Setting Blob storage aside which is primarily used for media files or documents, ATS provides three levels of data hierarchy (Table, PartitionKey, and RowKey) that can be accessed and navigated extremely efficiently.  However, anything beyond that will require custom code and CPU cycles of your compute nodes.  This is the key difference to work around.  It would be prudent to spend a significant amount of time modeling table storage with appropriate Blobs, Tables, PartitionKeys and RowKeys to accommodate for efficient data storage and retrieval strategies.  This will not only speed up your transactions and minimize the amount of data transferred in and out of ATS, but also reduce the burden on you compute nodes that will be required to manipulate data and directly translate into cost savings across the board.

Tip 5 – Data purging in ATS

Because you are charged for every gigabyte stored in ATS, it may be prudent to have a data purging strategy.  However, while it may seem like a straightforward problem in a world of relational databases, this is not the case with ATS.  Since ATS is not relational, deletion of each and every row from an ATS table requires two transactions.  In certain cases it may be possible to delete a single row using only one transaction.  Either way, this is extremely slow, inefficient and expensive.  A better way would be to partition a single table into multiple versions (e.g. Sales2010, Sales2011, Sales2012, etc.) and purge obsolete data by deleting a version of a table at a time.

Conclusion

Shift to cloud computing represents a major leap forward and enables almost everyone, from small companies to large enterprise, reduce their capital expenses, minimize time to market and significantly decrease support costs.  By investing even small effort into planning ahead, cloud computing can result in meaningful savings and benefits.

About CloudTweaks

Established in 2009, CloudTweaks is recognized as one of the leading authorities in connected technology information and services.

We embrace and instill thought leadership insights, relevant and timely news related stories, unbiased benchmark reporting as well as offer green/cleantech learning and consultive services around the world.

Our vision is to create awareness and to help find innovative ways to connect our planet in a positive eco-friendly manner.

In the meantime, you may connect with CloudTweaks by following and sharing our resources.

View All Articles

Sorry, comments are closed for this post.

Comic
When Sci-Fi Predictions Come To Fruition

When Sci-Fi Predictions Come To Fruition

Evolution of Technologies To paraphrase science fiction author Arthur C. Clark, those who make predictions about the future are either “considered conservative now and mocked later, or mocked now and proved right when they are no longer around to enjoy the acclaim.” The one thing we can be sure about, Clark ventured, is that “[the…

Facebook Hopes To Extend Internet Connectivity With Solar-Powered Drones

Facebook Hopes To Extend Internet Connectivity With Solar-Powered Drones

Facebook Inc (FB.O) said on Thursday it had completed a successful test flight of a solar-powered drone that it hopes will help it extend internet connectivity to every corner of the planet. Aquila, Facebook’s lightweight, high-altitude aircraft, flew at a few thousand feet for 96 minutes in Yuma, Arizona, Chief Executive Mark Zuckerberg wrote in…

When Will Women In Tech Become The Norm?

When Will Women In Tech Become The Norm?

Tech Diversity It is well known that the technology industry has been dominated by men, but it is also clear that the industry is working to change that. Diversity in the tech industry, especially where it applies to women in tech, has been a topic of discussion for years. Recently the Washington Technology Industry Association…

Four Keys For Telecoms Competing In A Digital World

Four Keys For Telecoms Competing In A Digital World

Competing in a Digital World Telecoms, otherwise largely known as Communications Service Providers (CSPs), have traditionally made the lion’s share of their revenue from providing pipes and infrastructure. Now CSPs face increased competition, not so much from each other, but with digital service providers (DSPs) like Netflix, Google, Amazon, Facebook, and Apple, all of whom…

Edtech and Virtual Reality – Exciting Learning Environment

Edtech and Virtual Reality – Exciting Learning Environment

Customizing Edutech Customized edtech learning solutions are becoming more commonplace as the education industry recognises their potential and begins transforming the traditional structures so as to incorporate innovative developments. From textbooks to tablets, chalkboards to virtual reality, edtech promises not only dynamic and exciting learning environments but better learning strategies and solutions. Virtual Reality and…

The Cloud Is Not Enough! Why Businesses Need Hybrid Solutions

The Cloud Is Not Enough! Why Businesses Need Hybrid Solutions

Why Businesses Need Hybrid Solutions Running a cloud server is no longer the novel trend it once was. Now, the cloud is a necessary data tier that allows employees to access vital company data and maintain productivity from anywhere in the world. But it isn’t a perfect system — security and performance issues can quickly…

Four Keys For Telecoms Competing In A Digital World

Four Keys For Telecoms Competing In A Digital World

Competing in a Digital World Telecoms, otherwise largely known as Communications Service Providers (CSPs), have traditionally made the lion’s share of their revenue from providing pipes and infrastructure. Now CSPs face increased competition, not so much from each other, but with digital service providers (DSPs) like Netflix, Google, Amazon, Facebook, and Apple, all of whom…

Connecting With Customers In The Cloud

Connecting With Customers In The Cloud

Customers in the Cloud Global enterprises in every industry are increasingly turning to cloud-based innovators like Salesforce, ServiceNow, WorkDay and Aria, to handle critical systems like billing, IT services, HCM and CRM. One need look no further than Salesforce’s and Amazon’s most recent earnings report, to see this indeed is not a passing fad, but…

The Cancer Moonshot: Collaboration Is Key

The Cancer Moonshot: Collaboration Is Key

Cancer Moonshot In his final State of the Union address in January 2016, President Obama announced a new American “moonshot” effort: finding a cure for cancer. The term “moonshot” comes from one of America’s greatest achievements, the moon landing. If the scientific community can achieve that kind of feat, then surely it can rally around…

Which Is Better For Your Company: Cloud-Based or On-Premise ERP Deployment?

Which Is Better For Your Company: Cloud-Based or On-Premise ERP Deployment?

Cloud-Based or On-Premise ERP Deployment? You know how enterprise resource management (ERP) can improve processes within your supply chain, and the things to keep in mind when implementing an ERP system. But do you know if cloud-based or on-premise ERP deployment is better for your company or industry? While cloud computing is becoming more and…

Cloud Computing – The Game Changer

Cloud Computing – The Game Changer

Global Cloud Index In October, Cisco released its Global Cloud Index (GCI) report for 2014-2019, projecting a near 3-fold growth of global data center traffic, with predictions that this traffic will reach 8.6 zettabytes (cloud data center traffic) and 10.4 zettabytes (total data center traffic) per year in 2019 and 80% of it will come…

Cloud Infographic – What Is The Internet of Things?

Cloud Infographic – What Is The Internet of Things?

What Is The Internet of Things? “We’re still in the first minutes of the first day of the Internet revolution.”  – Scott Cook The Internet of Things (IOT) and Smart Systems are based on the notions of Sensors, Connectivity, People and Processes. We are creating a new world to view and measure anything around us through…

Is The Fintech Industry The Next Tech Bubble?

Is The Fintech Industry The Next Tech Bubble?

The Fintech Industry Banks offered a wide variety of services such as payments, money transfers, wealth management, selling insurance, etc. over the years. While banks have expanded the number of services they offer, their core still remains credit and interest. Many experts believe that since banks offered such a wide multitude of services, they have…

Why Cloud Compliance Doesn’t Need To Be So Overly Complicated

Why Cloud Compliance Doesn’t Need To Be So Overly Complicated

Cloud Compliance  Regulatory compliance is an issue that has not only weighed heavily on the minds of executives, security and audit teams, but also today, even end users. Public cloud adds more complexity when varying degrees of infrastructure (depending on the cloud model) and data fall out of the hands of the company and into…

Cloud Infographic: IoT For Automotive Deconstructed

Cloud Infographic: IoT For Automotive Deconstructed

IoT For Automotive Deconstructed The IoT automotive industry is moving rapidly with many exciting growth opportunities available. We’ve written about some of the risks and benefits as well as some of the players involved. One thing for certain as that the auto industry is starting to take notice and we can expect the implementation of a…