Did AI blow up your cloud bill?

The term artificial intelligence was first used in a 1955 proposal for a study submitted by John McCarthy of Dartmouth College, Marvin Minsky of Harvard University, Nathaniel Rochester at IBM, and Claude Shannon at Bell Telephone Laboratories. This happened before I was born. I find it kind of nuts that AI was discussed long before we had the computing and storage power needed to make it work.

As a decision support analyst fresh out of college, I built early AI systems that were too expensive to operate, so they were only used in specialized circumstances. It was a niche technology. Because of its high operating costs, AI fell in popularity from the early 1980s until about five to seven years ago. Now, cloud computing’s on-demand consumption model and much better AI technology have substantially lowered the operating costs, and AI is back in focus for enterprise IT.

Public cloud providers are the driving force behind the current AI resurgence. Even though AI technology is now better optimized (and let’s just admit that it’s fun to play with), you need to fully understand the business value that it can return and acknowledge when the ROI is not there.

What’s more valuable to AI than cheaper and more powerful compute cycles? The fact that storage is a commodity. AI gets its power from learning data and understanding patterns in that learning data, not from cleverly written algorithms. The more data available to a learning model, the more focused the data becomes and the better knowledge or understanding it creates.

Despite its substantially lower operating costs and the potential value that AI and machine learning can bring to a business, the return falls short in many cases. 2022 was a year of huge cloud cost overruns. An enterprise’s misuse of cloud resources in general creates most cloud cost overruns. In some cases, this means choosing cloud AI/ML systems when more pragmatic alternatives could return more value.

Many AI/ML systems are much more expensive to maintain. Specialized skills are needed to build and deploy these systems and then to operate them. “Cloud AI” just means that the processing and data storage are outside of the enterprise. Massive amounts of general purpose and purpose-built data are needed to drive AI engines, and that data must be stored, managed, and secured ongoing. You must also deal with data compliance.

In many cases, the business has custom needs that require custom training data that isn’t part of the general-purpose transactional business database but is a one-off to support a specific need of the AI system. That means more storage, more labeling, more streaming, and more operational costs.

All of this may be worth it if there’s a strong business use case. In many instances, there isn’t. The easy availability of AI led to it being used where it’s not needed.

For example, a sound use case for AI might be a sales order entry system that leverages machine learning to determine recommendations that are automatically presented to customers ordering online. AI could increase sales and thus return business value. However, AI is used more often within traditional transactional systems where it only provides a minor advantage. An example of misuse would be to run AI to check for a valid shipping address to reduce shipping errors.

Remember, there are two sides to every AI use case. In the second example, the shipping savings could be a few thousand a month, which is good. But the cost of developing and operating the cloud-based AI system could be up to 20 times that amount per month, which is bad. There are on-demand solutions that don’t use AI but are as effective or more effective and can be had for a few hundred dollars per year.

The problem is gating. Cloud providers and consultants often recommend AI for use cases where it won’t provide the ROI needed for the business. It slips through if someone doesn’t ask the tougher questions or if a solid business case is never made.

It’s not a matter of whether AI works—it always works. It’s the misapplication of AI systems in the cloud that removes value from the business. Make that mistake often enough and the business will be no more.

I’m not pushing back on AI or AI in the cloud. I’ve made amazing applications that use AI concepts and technologies, and I will make many more in the future. This technology can do incredible things. However, as with any technology, AI has its place. We need to be more attentive to the ROI of its usage.