August 25, 2023

The Goldilocks Principle of Cloud Management: Striking the Ideal Balance

By Bharti Patel

It’s not an all-or-nothing proposition: How to strike the right balance with cloud

The pandemic swung the IT pendulum to the extreme as businesses rushed to the cloud.

Now organizations are seeing the bill come due, and they may be surprised at what they find.

While the cloud can be useful for some things, it is not the ideal match for every application and can become costly with scale. Cloud costs can at least double the infrastructure bills of companies operating at scale, put pressure on margins and weigh heavily on market cap, writes Andreessen Horowitz. And entrepreneur and popular blogger David Heinemeier Hansson reports that operating in the cloud can come with an “almost absurd premium.”

Most people would agree that you won’t get a decent return on investment out of the cloud if you just lift and shift your current architecture. But there’s an argument that you can do well with the cloud if you migrate to microservices. Yet, as Hansson notes, Amazon Prime itself dumped its microservices architecture, decreasing its cloud infrastructure costs by a staggering 90%.

Clearly, cloud costs have become a significant problem. However, the challenge with the cloud extends beyond the cost – it’s also about performance, security and data management.

Businesses must consider all four dimensions as they work on cloud optimization and striking the right balance between what they do in the public cloud and what they handle on premises.

Here’s why and how.

Analyze performance

Everybody is talking about ChatGPT, and many companies are launching efforts in the larger realm of artificial intelligence. When companies use large language models, they may upload all that data to the cloud. Enterprise applications such as SAP have similar workload requirements.

If you put that data in the cloud, you will have to pay an egress cost whenever you bring your data down. That’s expensive, but you might think you must pay that price for performance.

But data doesn’t need to live in the public cloud. Instead, you can store it “near-cloud” and still enjoy near sub-millisecond access to the cloud resources that you need.

If you adopt the right near-cloud solution, you will also get guaranteed availability. That’s dramatically better than what you will get from the cloud. No cloud provider is going to provide you with a 100% data availability guarantee. Cloud providers’ guarantees are so small that they are almost negligible. The guarantees really exist as compensation for not meeting cloud SLAs.

Address security, privacy and data access

Meta was recently hit with a $1.3 billion fine for moving European personal data to the U.S., which was in violation of the European Union’s General Data Protection Regulation (GDPR). Calling the move an “unjustified and unnecessary fine,” Meta blogged that it plans to appeal the ruling. The company added that it was “disappointed to have been singled out when using the same legal mechanism as thousands of other companies looking to provide services in Europe.”

This raises the question of whether other big, powerful companies think they also have the right to move people’s data around as they see fit. If that’s the case, whose data is safe?

Some organizations with compliance obligations are beginning to adopt “near-cloud” strategies that deliver nearly the same performance as cloud-native storage at a lower cost with the benefit of managing precisely where those data resources reside.

It’s up to you to decide how much ambiguity you can live with and what data policies to set. There may be certain types of data that should not leave the locked room inside your data center and other data that you are happy to make publicly available. Set policies for which data you want to keep on premises and which data you are comfortable storing in or near the cloud.

Also ensure that you have data discovery and visibility capabilities. This will enable you to understand what data is where so that you can effectively address compliance.

Understand data management

Data Merging Cloudtweaks Comic

Making sure data is secured, accessible and in the optimal location is part of the nuts and bolts of managing data. But such efforts should really be defined as storage management.

Data management, meanwhile, is more around the nature of the data. What data do you have? What do the bits and bytes of that data mean? Policy and compliance happen at that level.

Yet data management and storage management often get blurred into one. What some people refer to as data management is just table stakes data hygiene for storage management. You need to go the next layer up and do data management.

Both storage management and data management are important. You must do both. But be sure you understand that what some people are calling data management is just basic hygiene; they are not touching the data at all. Make sure you have the capabilities to catalog data, do analytics on it and otherwise address data management at every stage of the data value chain.

Act holistically

Many organizations adopted a “cloud first” strategy with good intentions of optimizing their utilization of infrastructure resources. In some cases, careful planning and understanding all of the tools available delivered extremely beneficial results. However, many more organizations have rightly scrutinized the cost-benefit of their “cloud-first” or “cloud-only” decisions.

Data storage infrastructure has quietly emerged as one area for extreme cloud optimization. Egress costs (downloading the data you freely uploaded to the public cloud) and replication costs (moving copies of data to another location for redundancy and protection) have significant cost implications. For many organizations, it can be the single greatest cloud cost.

Now that the cloud – and those that use it – have matured, everybody has learned a lot more about what works and what doesn’t. So, we can all be more careful about what to put where.

In deciding what to do, remember that there is more than one dimension to cloud optimization. Look at things holistically – and consider cost, performance, security and data management.

By Bharti Patel

Bharti Patel

Bharti Patel is SVP, Head of Engineering at Hitachi Vantara, spearheading the company’s cutting-edge data and infrastructure product offerings. She was previously CTO at Austin-based Alen, and spent +20 years at IBM leading large global R&D teams across ecommerce, mobility, client platforms, and data and storage management.
The Lighter Side Of The Cloud
Usman Choudhary

Navigating the Evolving Email Threat Landscape: A Conversation with VIPRE Security’s Usman Choudhary

Navigating the Evolving Email Threat Landscape In the ever-evolving digital landscape, email remains a favored [...]
Read more
Stacey Farrar

Six Things to Consider When Choosing Between Free and Paid Migration Tools

Choosing Between Free and Paid Migration Tools Microsoft recently decided to stop offering its free [...]
Read more
Vasilena Markova

DNS Attacks: Protecting Your Digital Assets with Advanced Security

DNS Attacks: Advanced Security In an increasingly interconnected digital world, the threat of DNS attacks [...]
Read more

Common Malware Anti-Analysis Techniques and How to Counter Them

Common Malware Anti-Analysis Techniques Malware analysis forms the backbone of proactive cybersecurity, making it possible [...]
Read more
Frank Kim

Frank Kim on Zero-Trust Architecture: Essential for Cloud Security

Zero-Trust Architecture In an interview with CloudTweaks, Frank Kim, a SANS Fellow and cloud security [...]
Read more
Arman Borghem

Exploring Digital Sovereignty with Arman Borghem: A Dive into Privacy and Compliance

Exploring Digital Sovereignty with Arman Borghem Welcome to today’s enlightening conversation on digital sovereignty and [...]
Read more
Unlock unparalleled exposure for your brand with CloudTweaks' premium sponsorship and advertising programs. Reach a global audience, amplify your message, and drive growth with our tailored solutions. Partner with us today and elevate your marketing strategy to new heights!
© 2024 CloudTweaks. All rights reserved.