Digital Transformation – Updated Metrics for the Cloud Era

Cloud Era Metrics

Undertaking digital transformation means also transforming how IT success is defined, including metrics that address business in the cloud. 

With up to 90% of budgets spent keeping the lights on, cost is always an important metric for IT. Yet, the cloud hype is also about improving performance — going beyond these time and budget basics as the obvious performance metrics, it’s the secondary effects of operating in the cloud that are often the culprits for under-performance and over-spend.

For example, a secondary-effect shortage of IT experts wielding skills in integration, interoperability and security can raise costs when an organization is unable to timely reskill current resources, forcing the rental of high-priced contractors to provide these ‘must-have’ skills for cloud projects or support. The occurrence of this type of unforeseen issue of business in the cloud can have a significant toll on overall impact.

Moving to cloud services is a journey as opposed to prior IT investments that measured success solely based on the ‘destinations’ of time, budget, and deliverables. There is no fixed set of directions for making your cloud journey; every cloud transformation means a different experience. Many enterprises begin their cloud journey with a lift and shift to Infrastructure as a Service (IaaS) that extends the life and value of existing investments. Gauging success with the cloud must include measuring these behind-the-scenes stops such as IaaS in addition to customer-facing stops along the way.

Consider the following as you rethink how to measure cloud services:

  • Performance of the cloud footprint as it expands more deeply into everyday use
  • Effectiveness of cloud customer services in improving user experiences
  • Impact of a dynamic work force on the ability to succeed at doing business in the cloud

Cloud Footprint

A shift in metrics from IT operational performance to outcome-based models is necessitated as your cloud footprint expands. Measure success by assessing how well solutions enable business capabilities (such as new features delivered) — rather than only whether operational requirements have been met — if consistent progress is being made towards achieving the outcomes. Project and operational deliverables are still important, but cloud metrics should also reflect time to deliver, multiple delivery cycles, and smaller-scoped work products that may not be 100% complete.

Business-based metrics are more important than ever. Historically, what CFO’s saw as “IT projects” lacked broader business involvement — this is problematic in the era of digital transformation where the consideration of moving to the cloud must be driven by business. Additionally, business should be defining the metrics that compare a pre- and post-cloud enterprise and if transformation is meeting overall corporate goals.

Financially, it is important to walk before you run. Make targeted SaaS investments to drive specific business impacts, rather than go to the cloud for cloud’s sake. Only purchase the amount of SaaS you need from vendors for your initial rollout and as you prove the concept, purchase more. This way, if you decide to end the cloud project, you are not stuck with subscriptions that never get used and that dilute metrics for ROI and optimization.

Data metrics are also important, particularly as inexpensive storage is making massive data collection possible. Mishandling of data stored in the cloud or thinly spread over a hybrid cloud/internally deployed portfolio can render it useless. Quality and availability of data are as important as quantity; be sure to measure data vectors (such as which data is used and how often) to show how well data is being leveraged to maintain or improve your competitive advantage.

Customer Services

As a fundamental part of the cloud is providing capabilities as services, there is a need to measure this from two different perspectives:

  1. The extent to which the cloud solution enables better internal planning and operations
  2. How the cloud solution improves the external customer experience.

Here, standard metrics such as web site clicks, sign-ups, and volume of digital sales are important, but it is the quality of a customer/user experience that will define the winners and losers in the cloud era. Clarifying the customer journey and which stage is most important to measure are key early steps. For example, metrics that demonstrate customer satisfaction and engagement levels for services received can be initial indicators of whether your cloud investments are paying off. Look at metrics such as ease of use, service usage volumes, and service outcome improvements over time.

Artificial intelligence (AI) is increasingly being used to lower service costs and streamline customer service. As AI permeates your cloud portfolio, establish metrics that demonstrate the health and well-being of the AI. Quality of the AI algorithms, effectiveness of AI responses, and ease of maintaining the AI knowledge library are examples of secondary metrics that should be measured over time.

Dynamic Workforce

Staffing to support the cloud can be one of the most difficult of the secondary effects to get right. Metrics here should assess three areas: culture, talent, and organizational structure. In the cloud era, a ‘work anywhere’ approach to staffing is rapidly becoming the new norm, which can be great for fleshing out missing expertise. Yet, cloud solutions demand persistent, interdisciplinary teams — measure how well yours stay together over time and whether they speak a common digital language that is necessary for design thinking and future-state journey mapping.

Pinterest Comic

Organizational changes necessary to embrace new ways of doing business don’t get any easier with the cloud. In addition to changing processes and systems, the business model itself likely will be evolving, as will the manner in which business and IT work together, defining and deploying technology. Measure your staff’s willingness and ability to change, as well as how well the business and IT are collaborating to develop and maintain cloud solutions.

While not an exhaustive list, here are sample metrics and vectors for cloud measurement:

CATEGORY

SAMPLE IT METRICS

SAMPLE BUSINESS METRICS

SAMPLE VECTORS TO MEASURE

Cloud Footprint

  • Outcome-based progress

  • Scalability effectiveness / agility

  • Speed/pace of iterative improvements and changes

  • Effectiveness of data usage

  • Integration effectiveness

  • Business capability enablement / improvement

  • Business involvement in solutioning

  • Ability to access, use cloud data

  • Business process integrity across solutions

  • Quality of data governance and data management

  • Interim deliverable progress and quality

  • Hardware and software assets retired

  • Time to market for solutions and enhancements

  • Increase/decrease in level of technical debt

  • Features delivered

  • Data usage by item and frequency

Customer Services

  • Web site operational performance

  • Service usage

  • Quality of AI algorithms

  • AI / Machine Learning effectiveness

  • Service outcome improvements

  • Customer/user experience – outcome improvement

  • Ease of use

  • AI response accuracy and effectiveness

  • Maturity of AI knowledge library

  • Customer happiness/ satisfaction

  • Net promoter score – customer loyalty

  • Customer effort score – customer experience

  • Client sentiment

  • Customer lifetime value – financial value of a customer

  • Click rates, sign-ups, volumes of digital sales

  • Milestones met / delivered on time

  • Performance – number of cases open and closed

  • Vendor-delivered patches, fixes, or features

  • Number of languages served / number of time zones covered

  • Resolution times

  • Cost to serve

  • Volume of asset or resource downloads

Dynamic Workforce

  • Team diversity – interdisciplinary strength (talent)

  • Team consistency (organizational structure)

  • Ability to adapt to change

  • Business and IT collaboration effectiveness

  • Organizational change effectiveness

  • Quality of governance over change

  • Cultural adaptation to the transformation

  • Resource experience and skills

  • Resource availability

  • Number of products and services covered

  • Resource utilization

  • Training effectiveness / error rates

  • Employee morale

When identifying cloud metrics, avoid the temptation to include too many measures. As many cloud metrics are ‘soft,’ meaning they are qualitative rather than quantitative, you may need to look for and consider new performance indicators. Ensure you are working towards creating co-valued (shared) metrics by collaborating across the business and IT to link metrics to key enterprise initiatives and strategies. Focus on the top five or ten that are most meaningful and strive for consistent evaluation and openness for adaption.

By Sebastian Grady

David Gevorkian

Website Accessibility: Compliancy, Laws and Best Practices

Key to Making Your Website Accessible The internet has changed the education sector in so many ways. With e-learning, more people around the globe are able to access high-quality education and advance their careers. E-learning ...
Ajay

Explainable Intelligence Part 3 – The Strategy for XAI

The Strategy for XAI It is not enough to say that something is true just because 'I know it’s true!' – we have to have some evidence or argument that gives a justification for our ...
Juan Pablo Perez Etchegoyen

7 Security and Compliance Considerations for Cloud-Based Business Applications  

Security and Compliance Considerations There’s no doubt on-premises deployments of mission-critical business applications provide more control over data as it resides within the four walls of an organization’s network infrastructure. However, businesses can no longer ...
Kaylamatthews

New Security Regulation – Cybersecurity Maturity Model Certification (CMMC)

Cybersecurity Maturity Model Certification Changes are on the horizon for the Department of Defense (DoD) and its contractors. Late last year, the DoD announced the Cybersecurity Maturity Model Certification (CMMC), which officially released in January. The ...
Anita Raj

Will there be a normal to go back to after COVID-19?

The COVID-19 Aftermath Until November last year, not one of us would have expected life to take such a dramatic turn in as short as 4-5 months. Yet here we are - confined to our ...
Kokumai

History, Current Status and Future Scenarios of Expanded Password System

Future Scenarios of Expanded Password System Passwords are so hard to manage that some people are urging the removal of passwords from digital identity altogether. What would happen, then, if the password is removed from our ...