Getting Value Out of GenAI

(OK-product-studio/Shutterstock)

As the initial excitement over generative AI starts to fade, some executives are beginning to question whether it will pan out in the long run. Others, however, are still quite bullish on GenAI’s potential to create value, provided it meets a few key requirements.

Goldman Sachs turned some heads recently with a report casting doubt on GenAI and raising uncomfortable questions about the potential for companies to see a return on their GenAI investments. Gartner also issued a report recently saying 30% of GenAI initiatives will be abandoned by 2025.

While there are some issues to be worked out around cost, transparency, and the best way to use it, there is still a big  upside to GenAI, says Karthik Sj, the general manager of AI at LogicMonitor ,a provider of IT observability and AIOps solutions.

“Putting in AI for the sake of AI, that is one reason why there’s a lack of adoption,” Sj says. “The trick is you have to narrow it down to the right use case, which is I would say, a high-value, high pain-point and also a high-volume thing, especially in areas where you’re seeing labor shortages, where humans just don’t want to do things.”

One such use case, you may be surprised to learn, is network and application monitoring. Each application is emitting its own signal these days, Sj says, so there is a plethora of alerts to look at. That can easily overwhelm human eyes.

“IT complexity has grown so much,” he says. “Today you have expert NOC engineers who are piecing this together and trying to interpret what exactly that means, and is this something I should pay attention to or not? It’s really hard to keep up.”

Companies must develop more sophisticated apps than Q&A chatbots to get real value from GenAI (sdecoret/Shutterstock)

Instead of asking the human NOC engineer to do the busy work of signing into multiple observability tools, normalizing the data, and then figuring out how they relate, one could task a GenAI tool with doing that work and setting the table for the human engineer or analyst to look at the assembled data and make a decision.

Beyond Chatbots

Finding the right use case is a crucial first step for getting value from GenAI. But ultimately, the potential to apply GenAI beyond applications such as chatbots and question and answering will be too hard to resist. The future of GenAI is enabling it to take actions, Kj says.

“The future is going to be about taking actions, like what if I could let the AI autonomously fix my router problem? Why does it need to give me a step by step? Why can’t it just go do it?” Kj tells Datanami in a recent interview. “There will be a shift towards automation. It’s not enough to just give answers.”

That is ultimately the plan at LogicMonitor, Kj says. Today it’s using GenAI to assist with summarization of log analysis, but in the future, the goal is to enable AI to begin to take some actions. The company will start small and eventually give customers the option to enable AI to automate a small percentage of possible actions.

The adoption of new technology like AI will happen in steps. Today, about 50% of LogicMonitor customers are using GenAI for things like log summarization. Customer sentiment so far is positive, which gives the company the incentive to keep investing in it.

“One of the things we’re getting a lot is, give me an opportunity to give feedback and use that [human] feedback in the loop,” he says. “[Another is] can you look at more data? I don’t just look at my observability data, but look at [other data]. So they want to go wide and also go deep.”

GenAI is one tool in the toolbox for companies like LogicMonitor and the customers they serve. GenAI may be good for summarization now and possibly taking some degree of autonomous action in the future, but it’s not the end-all, be-all. For instance, finding correlation between multiple data sets may be best done with a classic machine learning algorithm, so integration with other forms of AI will also help customers ultimately get more value out of GenAI.

“You don’t need LLMs for everything. I think the LLMs really are effective in [generating] human-readable summaries,” Kj says. “The other stuff that you need like correlation and all that, you don’t need generative AI for that. You can totally get the work done with, you know, causal AI or just statistical AI. That’s good enough. It gets the job done.”

Taking the Long View

The hype around GenAI has led to distortions in the marketplace. For instance, there has been a run on high-end GPUs that are necessary for training and running the biggest LLMs, which has resulted in a Nvidia achieving a $3 trillion market cap. During the run on GPUs, many large companies have hoarded GPUs to prevent their competitors from getting their hands on them. They also can be expensive to rent in the cloud.

“There’s definitely a lot of shelfware,” Kj says. “There’s a lot of AI washing going around and I think you’ve got to dig under the covers and say, hey, what is the AI you’re using? How are you using it? Many companies claim being AI powered just by calling it GPT, and that’s not AI-powered. AI-powered [refers] to this new class of applications that are built from the ground up.”

Adopting new technology like GenAI doesn’t eliminate the need to mind the details. Companies must get a return on their AI investments, or the dollars will be invested elsewhere. Ultimately, automation via AI will save hours’ worth of work per week, whether it’s in AIops or marketing or copywriting, Sj says.

How much time, exactly, will GenAI save companies in the short run? “I don’t think it’s 5%. I think that’s too pessimistic,” he says. “I’m absolutely sure it’s more than 5%. Is it 50%? Probably not. But I think at least it will be in the high range 15% to 20%. That’s what I’m seeing.

“As long as you can show ROI and this is a high pain-point problem,” he says, “you definitely are seeing value.”

Related Items:

Is the GenAI Bubble Finally Popping?

Gartner Warns 30% of GenAI Initiatives Will Be Abandoned by 2025