Cloud Era Metrics

Undertaking digital transformation means also transforming how IT success is defined, including metrics that address business in the cloud. 

With up to 90% of budgets spent keeping the lights on, cost is always an important metric for IT. Yet, the cloud hype is also about improving performance — going beyond these time and budget basics as the obvious performance metrics, it’s the secondary effects of operating in the cloud that are often the culprits for under-performance and over-spend.

For example, a secondary-effect shortage of IT experts wielding skills in integration, interoperability and security can raise costs when an organization is unable to timely reskill current resources, forcing the rental of high-priced contractors to provide these ‘must-have’ skills for cloud projects or support. The occurrence of this type of unforeseen issue of business in the cloud can have a significant toll on overall impact.

Moving to cloud services is a journey as opposed to prior IT investments that measured success solely based on the ‘destinations’ of time, budget, and deliverables. There is no fixed set of directions for making your cloud journey; every cloud transformation means a different experience. Many enterprises begin their cloud journey with a lift and shift to Infrastructure as a Service (IaaS) that extends the life and value of existing investments. Gauging success with the cloud must include measuring these behind-the-scenes stops such as IaaS in addition to customer-facing stops along the way.

Consider the following as you rethink how to measure cloud services:

• Performance of the cloud footprint as it expands more deeply into everyday use
• Effectiveness of cloud customer services in improving user experiences
• Impact of a dynamic work force on the ability to succeed at doing business in the cloud

Cloud Footprint

A shift in metrics from IT operational performance to outcome-based models is necessitated as your cloud footprint expands. Measure success by assessing how well solutions enable business capabilities (such as new features delivered) — rather than only whether operational requirements have been met — if consistent progress is being made towards achieving the outcomes. Project and operational deliverables are still important, but cloud metrics should also reflect time to deliver, multiple delivery cycles, and smaller-scoped work products that may not be 100% complete.

Business-based metrics are more important than ever. Historically, what CFO’s saw as “IT projects” lacked broader business involvement — this is problematic in the era of digital transformation where the consideration of moving to the cloud must be driven by business. Additionally, business should be defining the metrics that compare a pre- and post-cloud enterprise and if transformation is meeting overall corporate goals.

Financially, it is important to walk before you run. Make targeted SaaS investments to drive specific business impacts, rather than go to the cloud for cloud’s sake. Only purchase the amount of SaaS you need from vendors for your initial rollout and as you prove the concept, purchase more. This way, if you decide to end the cloud project, you are not stuck with subscriptions that never get used and that dilute metrics for ROI and optimization.

Data metrics are also important, particularly as inexpensive storage is making massive data collection possible. Mishandling of data stored in the cloud or thinly spread over a hybrid cloud/internally deployed portfolio can render it useless. Quality and availability of data are as important as quantity; be sure to measure data vectors (such as which data is used and how often) to show how well data is being leveraged to maintain or improve your competitive advantage.

Customer Services

As a fundamental part of the cloud is providing capabilities as services, there is a need to measure this from two different perspectives:

1. The extent to which the cloud solution enables better internal planning and operations
2. How the cloud solution improves the external customer experience.

Here, standard metrics such as web site clicks, sign-ups, and volume of digital sales are important, but it is the quality of a customer/user experience that will define the winners and losers in the cloud era. Clarifying the customer journey and which stage is most important to measure are key early steps. For example, metrics that demonstrate customer satisfaction and engagement levels for services received can be initial indicators of whether your cloud investments are paying off. Look at metrics such as ease of use, service usage volumes, and service outcome improvements over time.

Artificial intelligence (AI) is increasingly being used to lower service costs and streamline customer service. As AI permeates your cloud portfolio, establish metrics that demonstrate the health and well-being of the AI. Quality of the AI algorithms, effectiveness of AI responses, and ease of maintaining the AI knowledge library are examples of secondary metrics that should be measured over time.

Dynamic Workforce

Staffing to support the cloud can be one of the most difficult of the secondary effects to get right. Metrics here should assess three areas: culture, talent, and organizational structure. In the cloud era, a ‘work anywhere’ approach to staffing is rapidly becoming the new norm, which can be great for fleshing out missing expertise. Yet, cloud solutions demand persistent, interdisciplinary teams — measure how well yours stay together over time and whether they speak a common digital language that is necessary for design thinking and future-state journey mapping.

Organizational changes necessary to embrace new ways of doing business don’t get any easier with the cloud. In addition to changing processes and systems, the business model itself likely will be evolving, as will the manner in which business and IT work together, defining and deploying technology. Measure your staff’s willingness and ability to change, as well as how well the business and IT are collaborating to develop and maintain cloud solutions.

While not an exhaustive list, here are sample metrics and vectors for cloud measurement:

When identifying cloud metrics, avoid the temptation to include too many measures. As many cloud metrics are ‘soft,’ meaning they are qualitative rather than quantitative, you may need to look for and consider new performance indicators. Ensure you are working towards creating co-valued (shared) metrics by collaborating across the business and IT to link metrics to key enterprise initiatives and strategies. Focus on the top five or ten that are most meaningful and strive for consistent evaluation and openness for adaption.

By Sebastian Grady