Digital Transformation – Updated Metrics for the Cloud Era

Cloud Era Metrics

Undertaking digital transformation means also transforming how IT success is defined, including metrics that address business in the cloud. 

With up to 90% of budgets spent keeping the lights on, cost is always an important metric for IT. Yet, the cloud hype is also about improving performance — going beyond these time and budget basics as the obvious performance metrics, it’s the secondary effects of operating in the cloud that are often the culprits for under-performance and over-spend.

For example, a secondary-effect shortage of IT experts wielding skills in integration, interoperability and security can raise costs when an organization is unable to timely reskill current resources, forcing the rental of high-priced contractors to provide these ‘must-have’ skills for cloud projects or support. The occurrence of this type of unforeseen issue of business in the cloud can have a significant toll on overall impact.

Moving to cloud services is a journey as opposed to prior IT investments that measured success solely based on the ‘destinations’ of time, budget, and deliverables. There is no fixed set of directions for making your cloud journey; every cloud transformation means a different experience. Many enterprises begin their cloud journey with a lift and shift to Infrastructure as a Service (IaaS) that extends the life and value of existing investments. Gauging success with the cloud must include measuring these behind-the-scenes stops such as IaaS in addition to customer-facing stops along the way.

Consider the following as you rethink how to measure cloud services:

  • Performance of the cloud footprint as it expands more deeply into everyday use
  • Effectiveness of cloud customer services in improving user experiences
  • Impact of a dynamic work force on the ability to succeed at doing business in the cloud

Cloud Footprint

A shift in metrics from IT operational performance to outcome-based models is necessitated as your cloud footprint expands. Measure success by assessing how well solutions enable business capabilities (such as new features delivered) — rather than only whether operational requirements have been met — if consistent progress is being made towards achieving the outcomes. Project and operational deliverables are still important, but cloud metrics should also reflect time to deliver, multiple delivery cycles, and smaller-scoped work products that may not be 100% complete.

Business-based metrics are more important than ever. Historically, what CFO’s saw as “IT projects” lacked broader business involvement — this is problematic in the era of digital transformation where the consideration of moving to the cloud must be driven by business. Additionally, business should be defining the metrics that compare a pre- and post-cloud enterprise and if transformation is meeting overall corporate goals.

Financially, it is important to walk before you run. Make targeted SaaS investments to drive specific business impacts, rather than go to the cloud for cloud’s sake. Only purchase the amount of SaaS you need from vendors for your initial rollout and as you prove the concept, purchase more. This way, if you decide to end the cloud project, you are not stuck with subscriptions that never get used and that dilute metrics for ROI and optimization.

Data metrics are also important, particularly as inexpensive storage is making massive data collection possible. Mishandling of data stored in the cloud or thinly spread over a hybrid cloud/internally deployed portfolio can render it useless. Quality and availability of data are as important as quantity; be sure to measure data vectors (such as which data is used and how often) to show how well data is being leveraged to maintain or improve your competitive advantage.

Customer Services

As a fundamental part of the cloud is providing capabilities as services, there is a need to measure this from two different perspectives:

  1. The extent to which the cloud solution enables better internal planning and operations
  2. How the cloud solution improves the external customer experience.

Here, standard metrics such as web site clicks, sign-ups, and volume of digital sales are important, but it is the quality of a customer/user experience that will define the winners and losers in the cloud era. Clarifying the customer journey and which stage is most important to measure are key early steps. For example, metrics that demonstrate customer satisfaction and engagement levels for services received can be initial indicators of whether your cloud investments are paying off. Look at metrics such as ease of use, service usage volumes, and service outcome improvements over time.

Artificial intelligence (AI) is increasingly being used to lower service costs and streamline customer service. As AI permeates your cloud portfolio, establish metrics that demonstrate the health and well-being of the AI. Quality of the AI algorithms, effectiveness of AI responses, and ease of maintaining the AI knowledge library are examples of secondary metrics that should be measured over time.

Dynamic Workforce

Staffing to support the cloud can be one of the most difficult of the secondary effects to get right. Metrics here should assess three areas: culture, talent, and organizational structure. In the cloud era, a ‘work anywhere’ approach to staffing is rapidly becoming the new norm, which can be great for fleshing out missing expertise. Yet, cloud solutions demand persistent, interdisciplinary teams — measure how well yours stay together over time and whether they speak a common digital language that is necessary for design thinking and future-state journey mapping.

Pinterest Comic

Organizational changes necessary to embrace new ways of doing business don’t get any easier with the cloud. In addition to changing processes and systems, the business model itself likely will be evolving, as will the manner in which business and IT work together, defining and deploying technology. Measure your staff’s willingness and ability to change, as well as how well the business and IT are collaborating to develop and maintain cloud solutions.

While not an exhaustive list, here are sample metrics and vectors for cloud measurement:

CATEGORY

SAMPLE IT METRICS

SAMPLE BUSINESS METRICS

SAMPLE VECTORS TO MEASURE

Cloud Footprint

  • Outcome-based progress

  • Scalability effectiveness / agility

  • Speed/pace of iterative improvements and changes

  • Effectiveness of data usage

  • Integration effectiveness

  • Business capability enablement / improvement

  • Business involvement in solutioning

  • Ability to access, use cloud data

  • Business process integrity across solutions

  • Quality of data governance and data management

  • Interim deliverable progress and quality

  • Hardware and software assets retired

  • Time to market for solutions and enhancements

  • Increase/decrease in level of technical debt

  • Features delivered

  • Data usage by item and frequency

Customer Services

  • Web site operational performance

  • Service usage

  • Quality of AI algorithms

  • AI / Machine Learning effectiveness

  • Service outcome improvements

  • Customer/user experience – outcome improvement

  • Ease of use

  • AI response accuracy and effectiveness

  • Maturity of AI knowledge library

  • Customer happiness/ satisfaction

  • Net promoter score – customer loyalty

  • Customer effort score – customer experience

  • Client sentiment

  • Customer lifetime value – financial value of a customer

  • Click rates, sign-ups, volumes of digital sales

  • Milestones met / delivered on time

  • Performance – number of cases open and closed

  • Vendor-delivered patches, fixes, or features

  • Number of languages served / number of time zones covered

  • Resolution times

  • Cost to serve

  • Volume of asset or resource downloads

Dynamic Workforce

  • Team diversity – interdisciplinary strength (talent)

  • Team consistency (organizational structure)

  • Ability to adapt to change

  • Business and IT collaboration effectiveness

  • Organizational change effectiveness

  • Quality of governance over change

  • Cultural adaptation to the transformation

  • Resource experience and skills

  • Resource availability

  • Number of products and services covered

  • Resource utilization

  • Training effectiveness / error rates

  • Employee morale

When identifying cloud metrics, avoid the temptation to include too many measures. As many cloud metrics are ‘soft,’ meaning they are qualitative rather than quantitative, you may need to look for and consider new performance indicators. Ensure you are working towards creating co-valued (shared) metrics by collaborating across the business and IT to link metrics to key enterprise initiatives and strategies. Focus on the top five or ten that are most meaningful and strive for consistent evaluation and openness for adaption.

By Sebastian Grady

Gilad David Maayan

Accessing (HPC) High Performance Computing

HPC in the Cloud Big data and Machine Learning (ML) can provide businesses with incredible insights and an innovative edge. However, to properly analyze the ...
Matt Holleran

Cloud marketplaces give startups a leg up – Part 2

Cloud Marketplaces In my last post, Cloud Platforms, Marketplaces, and Startups Part One, I examined the proliferation of partner ecosystems within the cloud software business, ...
Tunio Zafer

Questions To Ask Every Cloud Storage Provider

Cloud Storage Provider Questions As with many new technologies, attitudes toward cloud storage vary. Telephones were immobile; wearables perhaps unwarranted. And now, the global cloud ...
Thomas Franklin

Future of Stock Markets : Raising Capital Through ICO is 10x cheaper and 20x easier

Future of Stock Markets: Raising Capital Through ICO How blockchain will replace the stock markets as we know them today. Welcome to the future. It’s ...
Anita Raj

Can the cloud handle the streaming explosion caused by the pandemic?

The Streaming Digital Explosion From the time the coronavirus forced the global community to stay at home, a whopping 16 million people have newly subscribed ...