Digital Transformation – Updated Metrics for the Cloud Era

Cloud Era Metrics

Undertaking digital transformation means also transforming how IT success is defined, including metrics that address business in the cloud. 

With up to 90% of budgets spent keeping the lights on, cost is always an important metric for IT. Yet, the cloud hype is also about improving performance — going beyond these time and budget basics as the obvious performance metrics, it’s the secondary effects of operating in the cloud that are often the culprits for under-performance and over-spend.

For example, a secondary-effect shortage of IT experts wielding skills in integration, interoperability and security can raise costs when an organization is unable to timely reskill current resources, forcing the rental of high-priced contractors to provide these ‘must-have’ skills for cloud projects or support. The occurrence of this type of unforeseen issue of business in the cloud can have a significant toll on overall impact.

Moving to cloud services is a journey as opposed to prior IT investments that measured success solely based on the ‘destinations’ of time, budget, and deliverables. There is no fixed set of directions for making your cloud journey; every cloud transformation means a different experience. Many enterprises begin their cloud journey with a lift and shift to Infrastructure as a Service (IaaS) that extends the life and value of existing investments. Gauging success with the cloud must include measuring these behind-the-scenes stops such as IaaS in addition to customer-facing stops along the way.

Consider the following as you rethink how to measure cloud services:

  • Performance of the cloud footprint as it expands more deeply into everyday use
  • Effectiveness of cloud customer services in improving user experiences
  • Impact of a dynamic work force on the ability to succeed at doing business in the cloud

Cloud Footprint

A shift in metrics from IT operational performance to outcome-based models is necessitated as your cloud footprint expands. Measure success by assessing how well solutions enable business capabilities (such as new features delivered) — rather than only whether operational requirements have been met — if consistent progress is being made towards achieving the outcomes. Project and operational deliverables are still important, but cloud metrics should also reflect time to deliver, multiple delivery cycles, and smaller-scoped work products that may not be 100% complete.

Business-based metrics are more important than ever. Historically, what CFO’s saw as “IT projects” lacked broader business involvement — this is problematic in the era of digital transformation where the consideration of moving to the cloud must be driven by business. Additionally, business should be defining the metrics that compare a pre- and post-cloud enterprise and if transformation is meeting overall corporate goals.

Financially, it is important to walk before you run. Make targeted SaaS investments to drive specific business impacts, rather than go to the cloud for cloud’s sake. Only purchase the amount of SaaS you need from vendors for your initial rollout and as you prove the concept, purchase more. This way, if you decide to end the cloud project, you are not stuck with subscriptions that never get used and that dilute metrics for ROI and optimization.

Data metrics are also important, particularly as inexpensive storage is making massive data collection possible. Mishandling of data stored in the cloud or thinly spread over a hybrid cloud/internally deployed portfolio can render it useless. Quality and availability of data are as important as quantity; be sure to measure data vectors (such as which data is used and how often) to show how well data is being leveraged to maintain or improve your competitive advantage.

Customer Services

As a fundamental part of the cloud is providing capabilities as services, there is a need to measure this from two different perspectives:

  1. The extent to which the cloud solution enables better internal planning and operations
  2. How the cloud solution improves the external customer experience.

Here, standard metrics such as web site clicks, sign-ups, and volume of digital sales are important, but it is the quality of a customer/user experience that will define the winners and losers in the cloud era. Clarifying the customer journey and which stage is most important to measure are key early steps. For example, metrics that demonstrate customer satisfaction and engagement levels for services received can be initial indicators of whether your cloud investments are paying off. Look at metrics such as ease of use, service usage volumes, and service outcome improvements over time.

Artificial intelligence (AI) is increasingly being used to lower service costs and streamline customer service. As AI permeates your cloud portfolio, establish metrics that demonstrate the health and well-being of the AI. Quality of the AI algorithms, effectiveness of AI responses, and ease of maintaining the AI knowledge library are examples of secondary metrics that should be measured over time.

Dynamic Workforce

Staffing to support the cloud can be one of the most difficult of the secondary effects to get right. Metrics here should assess three areas: culture, talent, and organizational structure. In the cloud era, a ‘work anywhere’ approach to staffing is rapidly becoming the new norm, which can be great for fleshing out missing expertise. Yet, cloud solutions demand persistent, interdisciplinary teams — measure how well yours stay together over time and whether they speak a common digital language that is necessary for design thinking and future-state journey mapping.

Pinterest Comic

Organizational changes necessary to embrace new ways of doing business don’t get any easier with the cloud. In addition to changing processes and systems, the business model itself likely will be evolving, as will the manner in which business and IT work together, defining and deploying technology. Measure your staff’s willingness and ability to change, as well as how well the business and IT are collaborating to develop and maintain cloud solutions.

While not an exhaustive list, here are sample metrics and vectors for cloud measurement:

CATEGORY

SAMPLE IT METRICS

SAMPLE BUSINESS METRICS

SAMPLE VECTORS TO MEASURE

Cloud Footprint

  • Outcome-based progress

  • Scalability effectiveness / agility

  • Speed/pace of iterative improvements and changes

  • Effectiveness of data usage

  • Integration effectiveness

  • Business capability enablement / improvement

  • Business involvement in solutioning

  • Ability to access, use cloud data

  • Business process integrity across solutions

  • Quality of data governance and data management

  • Interim deliverable progress and quality

  • Hardware and software assets retired

  • Time to market for solutions and enhancements

  • Increase/decrease in level of technical debt

  • Features delivered

  • Data usage by item and frequency

Customer Services

  • Web site operational performance

  • Service usage

  • Quality of AI algorithms

  • AI / Machine Learning effectiveness

  • Service outcome improvements

  • Customer/user experience – outcome improvement

  • Ease of use

  • AI response accuracy and effectiveness

  • Maturity of AI knowledge library

  • Customer happiness/ satisfaction

  • Net promoter score – customer loyalty

  • Customer effort score – customer experience

  • Client sentiment

  • Customer lifetime value – financial value of a customer

  • Click rates, sign-ups, volumes of digital sales

  • Milestones met / delivered on time

  • Performance – number of cases open and closed

  • Vendor-delivered patches, fixes, or features

  • Number of languages served / number of time zones covered

  • Resolution times

  • Cost to serve

  • Volume of asset or resource downloads

Dynamic Workforce

  • Team diversity – interdisciplinary strength (talent)

  • Team consistency (organizational structure)

  • Ability to adapt to change

  • Business and IT collaboration effectiveness

  • Organizational change effectiveness

  • Quality of governance over change

  • Cultural adaptation to the transformation

  • Resource experience and skills

  • Resource availability

  • Number of products and services covered

  • Resource utilization

  • Training effectiveness / error rates

  • Employee morale

When identifying cloud metrics, avoid the temptation to include too many measures. As many cloud metrics are ‘soft,’ meaning they are qualitative rather than quantitative, you may need to look for and consider new performance indicators. Ensure you are working towards creating co-valued (shared) metrics by collaborating across the business and IT to link metrics to key enterprise initiatives and strategies. Focus on the top five or ten that are most meaningful and strive for consistent evaluation and openness for adaption.

By Sebastian Grady

Rajesh Khanna
Implement Hyperautomation to Scale Automation Programs by 3X Most Digital Service Providers (DSPs) struggle to accelerate their path to Hyperautomation due to the complex processes with legacy systems and applications. Although Robotic Process Automation (RPA) plays a ...
Brian Rue
What’s Holding DevOps Back And How Developers and Businesses Can Vault Forward to Improve and Succeed Developers spend a lot of valuable time – sometimes after being woken up in the middle of the night ...
Yuliya Melnik
Heroku or AWS Cloud infrastructures are gradually starting to penetrate into an increasing number of areas and various businesses. And this is not surprising because such a ploy allows you to improve internal processes, protect ...
JK Chelladurai
Usage-Based Pricing We are now in an era where many businesses are flipping their business model and shifting from subscription-based pricing to usage-based models, to better cater to the modern ‘pay-as-you-consume’ buyer. So what exactly ...
Matrix
When sci-fi films like Tom Cruise’s Oblivion depict humans living in the clouds, we imagine that humanity might one day leave our primitive dwellings attached to the ground and ascend to floating castles in the ...

SECURITY TRAINING

  • Isc2

    ISC2

    (ISC)² provides IT training, certifications, and exams that run online, on your premises, or in classrooms. Self-study resources are available. You can also train groups of 10 or more of your employees. If you want a job in cybersecurity, this is the route to take.

  • App Academy

    App Academy

    Immersive software engineering programs. No experience required. Pay $0 until you're hired. Join an online info session to learn more

  • Cybrary

    Cybrary

    CYBRARY Open source Cyber Security learning. Free for everyone, forever. The world's largest cyber security community. Cybrary provides free IT training and paid IT certificates. Courses for beginners, intermediates, and advanced users are available.

  • Plural Site

    Pluralsite

    Pluralsight provides online courses on popular programming languages and developer tools. Other courses cover fields such as IT security best practices, server infrastructure, and virtualization.