Cloud-based generative AI won’t be cheap

Even though many IT budgets are down, and belt tightening seems to be the clear trend, next year many enterprises are preparing for a rush to generative AI that they are not ready to pay for. It’s time to start thinking about how we’re going to make this work, and how cloud computing can be of assistance.

AI-driven supply chains, AI-driven manufacturing, AI-driven healthcare are all business cases that are on the table. Industries are clamoring for them and for good reason. The value that generative AI can bring (or at least what’s been bantered about) is unheard of compared to any technology trend I’ve seen in my long career. I understand why those predictions are being made.

Extinction-level events coming? 

This could be an extinction-level event for businesses that choose to ignore the potential value of this technology for their industry and their businesses. For instance, manufacturers with supply chains that approach 90% efficiency because of generative AI technology will have a huge leg up on those that operate at 60% efficiency. They will be able to provide faster production schedules, better customer experiences, better employee experiences, and higher quality at lower costs and thus prices. All this with their new ability to create near-perfect decisions, carried out automatically with all systems participating. At least that’s the dream.

Of course, we’re never going to get to that level as fast as we’d like, even with access to this technology—or any technology for that matter. However, it’s clear that many things are likely to change, and many companies will be left behind if they don’t keep up. Depending on their industry and business models, many may go extinct if they are unable to leverage cloud-based generative AI effectively and in time. 

Who pays for this stuff?

From where I’m sitting, it’s strange that we’re talking about budgets when most enterprises have yet to understand the value of this technology. But some do, and those companies are preparing for some major priority shifting of IT. Generative AI will require huge piles of cash. Where does that come from?

You must remember that IT, for most companies, is not a profit center, it’s a cost center. These costs are allocated throughout the company based on size, usage, productivity, and other ways of apportioning spending budgets. Indeed, we’re just getting a handle on cloud spending now with the rise of finops, which can divide costs fairly within companies.

The problem with the rise of generative AI, which is going to be just another group of cloud services that hog storage and compute services, is that a great deal of development investment will need to occur. In the past, those investments were also allocated among departments, normally equally if they all benefited from the new applications. For instance, cloud-based ERP costs, installation, and operations may have been split across all departments.

The reality of generative AI is that it’s going to be very expensive, You’ll need CPUs, GPUs, and storage systems, and you’ll need more than you think. This leads to an inflection point in innovation and spending for those companies that are all-in on generative AI and believe the story that I told earlier. 

I’m seeing the preplanning debates now, in terms of which budgets will fund the mother of all development and operations projects. I suggest we learn from what we did wrong with many initial cloud computing projects: namely, calling it an expense and doing cloud migration and development as half-measures. 

Too many companies made too many mistakes that killed the value they could have gotten from cloud computing. They relied too heavily on lift and shift, ignoring better paths to more cost-effective cloud infrastructure. They thought cloud computing was a cost-savings measure and not an investment. Enterprises are paying for that now as they must loop back and fix issues that should have been corrected initially. In other words, they are migrating the same workloads to the cloud twice.

My advice is to treat generative AI for what it really is: a strategic investment that may define the value of the company at some point. Don’t nickel and dime your way into this technology. Do it right the first time. I’m sure there won’t be a second chance with this one.