Waste Not, Want Not

(pathdoc/Shutterstock)

Cloud spend is at an all-time high. The latest forecast from Gartner backs that up with predictions that end-user spending worldwide on public cloud services will grow almost 21 percent this year to exceed $591 billion—a significant jump from last year’s estimated spend of $490 billion. That’s sobering news to CFOs and CIOs certainly, but perhaps the real kicker lies in the fact that a recent study from Flexera found that approximately 28% of cloud spend is going to waste.

Applying Gartner’s estimated dollar spend, that translates into almost $168 billion of waste—the most in any year, ever. That’s a staggering sum and explains why managing cloud spend was listed as the top cloud challenge by 82% of survey respondents.

The good news is that the percentage of waste compared to overall spend is down from 32 percent last year. The bad news is that companies tend to underestimate waste. It’s unsurprising then that in the FinOps Foundation’s State of FinOps 2023 survey, Reducing Waste or Unused Resources moved up the ranks to tie with Getting to Unit Economics (a new challenge for 2023) and Organizational Adoption of FinOps, among the top five challenges for FinOps practitioners.

The dual specters of cloud spend and waste coupled with strong economic headwinds is putting increased pressure on companies to become even more efficient. CIOs are increasingly being held accountable for their data teams’ cloud spend, and with cloud budgets continuing to make up the largest portion of IT spend, it follows that if overall IT budgets shrink, so too will cloud budgets.

Time to rein in your cloud costs (Body Stock/Shutterstock)

Trim the Sails

With data management as the fastest growing segment of public cloud spending, data teams need to get a handle on cloud spend yesterday. No wonder they’re increasingly turning to the practices of DataOps and FinOps (DataFinOps) as a means of rightsizing cloud data resources and managing costs associated with cloud data analytics.

Using the following best practices—each of which is rooted in the principles of DataOps and FinOps—enables companies to accelerate innovation while simultaneously reducing cloud waste.

• Know what you are using (and spending). A poor understanding of cloud consumption can translate to budget overages. In tenuous economic times, inadvertent overspend can break a company, making it essential that data teams get deep visibility into job, cluster, pipeline and user spending,  resource utilization, and efficiency metrics. Continuous consumption reporting provides additional insight into cloud usage and near real-time feedback for more efficient cloud spend.
• Identify overprovisioned cloud resources. Redeploying underutilized cloud investments enables teams to deliver more business value without increasing spend. To do so you must first find out where cloud data resources are overprovisioned and/or underutilized. Is it a department? A project? An individual? An application? The increased collaboration and visibility that a FinOps practice brings shines a light into cloud resources, usage, and costs and consequently leads to more informed decisions about cloud allocation.
• Nip it in the bud. One way to prevent and catch cloud data cost anomalies before they spiral out of control is with the implementation of code reviews, automation, and cost-monitoring best practices. Orchestrate workloads for faster cluster startup times, run jobs/pipelines after hours and during times of low demand to take advantage of spot pricing to get off-peak cloud computing rates, and, for companies in the midst of their digital transformation, move only optimized, high-value projects to the cloud as a way to deliver increased business value while simultaneously controlling cloud spend.
  

  Unused resources pile up in clouds (Nightman1965/Shutterstock)

• The AI advantage. Improved AI and machine learning solutions, while still a long way from being self-healing, can provide recommendations that will help your team to improve their cloud data spend efficiency. These technologies should be used to implement guardrails that will, in turn, alert you to potential overspend and provide actionable steps to optimize jobs and preemptively prevent budget overruns.
• Cost-conscious culture. Change up your corporate culture and consider implementing a FinOps practice as a means of reducing waste and getting more cloud bang for your buck. Successfully instituting a FinOps practice requires getting company-wide, multi-disciplinary buy-in where everyone takes ownership of their cloud usage and spend, thereby facilitating cost savings. A strong FinOps practice enables companies to make—not just save—money.

Cloud Waste Mountains

As companies look to weather economic headwinds, data leaders are advised to look for opportunities to redeploy overprovisioned and underutilized cloud spend.

Utilizing a collaborative FinOps approach can help data teams realize returns on their data stack investments while simultaneously freeing them to innovate.

John Steinbeck is credited with saying that the mountains of things we throw away are greater than the things we use. Don’t let your cloud budget be something you inadvertently toss away.

About the author: Clinton Ford, DataOps champion for Unravel Data, is passionate about unifying data science, engineering, and business. Clinton has launched data and AI cloud services, working with global customers in a variety of technical and management positions, including program management and finance at Microsoft, product marketing and product management at AWS, and cloud partner marketing at Databricks. He holds a B.S. in Computer Science from Brigham Young University and an MBA from the University of Washington. He can be reached at [email protected].

Related Items:

The High Costs of Going Cloud-Native

The Cloud Is Great for Data, Except for Those Super High Costs

Bringing Cloud Data Costs Under Control