Demand Builds for Third-Party GenAI Development Services

(everything possible/Shutterstock)

The emergence of generative AI is easily the biggest tech story of 2023, as products like ChatGPT have captured the imaginations of consumers and business leaders alike. We’re currently in the experimentation phase of GenAI adoption. But as companies look to go into production with GenAI, they will seek the services of external developers and systems integrators, including companies like Stellar AI, which recently came out of stealth.

Projected spending on GenAI is set to explode in the coming years. According to a recent IDC estimate, $16 billion will be spent on GenAI in 2023, rising to $143 billion by 2027, a 73.3 compound annual growth rate (CAGR). To put that in perspective, that’s twice the CAGR of the overall AI sector over that time, and 13 times greater than general purpose IT spending.

Much of this spending will go toward shrink-wrapped software and services, of course. OpenAI currently has the lion’s share of the nascent GenAI market, and is charging people to access to the large language models that underly ChatGPT, including GPT-3.5 and GPT-4, via its API. Other vendors are also selling access to their proprietary LLMs via APIs, which will continue to be a popular business model for customers that don’t want to get their hands dirty and want a quick and easy way to tap into the powers of GenAI.

While training AI models is easier than the early days of deep learning, building GenAI applications still requires quite a bit of technological sophistication across a range of disciplines. Beyond the data science of AI training and fine-tuning, there is data engineering work to ensure the data is ready to train an AI model. There may be vector databases, prompting tools, and retrieval-augmented generation (RAG) systems to set up. There are initial infrastructure requirements, and there are more requirements to scale a GenAI app in production. And then there are the business and financial questions, to say nothing about questions of ethics, safety, and regulation.

Consultants will play a big roll in GenAI development (Pixel-Shot/Shutterstock)

When you add up all these requirements, IDC sees would-be GenAI adopters seeking the assistance of experienced consultants to help shepherd AI into production.

“Because GenAI is still maturing as technology and is in the nascent stages of adoption by enterprises, metrics are not standardized and formalized,” the company wrote in a recent blog. “For these reasons, it’s a good idea to seek advice, project management, and implementation expertise from business and IT consultancies that have experience with AI and organizational change.”

Stellar Out of Stealth

One of the newer consultancies swimming in the GenAI waters is an Indianapolis, Indiana outfit called Stellar AI. Founded by Silicon Valley veterans Unmesh Kulkarni, Zach Linder, and Brett Flinchum, the company recently came out of stealth with a plan to help businesses develop and scale their GenAI applications.

Kulkarni says AI’s slow simmer turned into a rolling boil, which signaled that the time was right to launch Stellar as an AI consultancy.

“I have been doing AI and ML for the last eight to 10 years, but with the language models, in particular when language models became LLMs, the magic started happening,” Kulkarni said. “You now have a system that you can interact with like a human and it’s kind of breaking that Turing Test barrier now. So Brett, Zach, myself, and some of our investors gathered and said this is a really big opportunity.”

While the ready availability of huge and sophisticated LLMs like GPT-4 has lowered the technical barrier to using deep learning approaches, Stellar recognizes that there’s still quite a lot of work to do to stand up a GenAI application. That’s why the consultancy spends time to do a methodical review of customers and their AI goals.

“The first stage of engagement is where we say, let us come in and take a look at your environment and really let’s have a conversation about whether you should even do this,” Kulkarni said. “That answer is generally yes, but how should we go about that. Let’s not jump in and start writing code.”

Determining whether there will be a good return on a GenAI investment can help avoid painful lessons down the line. Stellar also strives to help the client understand the data security, governance, data lineage aspects of building and running a GenAI system, which have always been part of their AI engagements at previous companies. “That’s our background,” Kulkarni said. “That’s our expertise.”

Unmesh Kulkarni is CTO and co-founder of Stellar AI

The company also looks at a client’s existing machine learning projects, whether it’s traditional ML like logistic regression models or SVMs or deep learning, such as recurrent neural networks or transformer networks (which LLMs are based on). They’ll look at the data warehousing environment, and whatever unstructured documents–such as PDFs, sales proposals, FAQs, or legal documents–that are used to train GenAI models.

At that point, if Stellar has identified a suitable opportunity for the client, then they’ll go ahead with the project. Stellar has developed its own frameworks that can help the client get a proof of concept up and running quite quickly; the actual coding part isn’t the bottleneck in GenAI projects. Stellar helps these clients connect the dots in GenAI so they can make a decision to make more investments or not.

“There are companies that are basically saying look, I want to just go and experiment.  They’ve spun up these proof of concept teams and they are just experimenting,” Kulkarni said. “I think that’s great. But they don’t necessarily have the breadth or the talent to actually go in the right direction. They’re spinning a lot of cycles and we can help them…quickly get to the right model.”

Catering to Privacy and Control

Stellar came out of stealth in August, but it already has clients in the medical device industry, law, manufacturing, and healthcare. The company is keen to capitalize on the excitement around GenAI and the expected surge of spending to help customers build GenAI apps that not only deliver value, but do so without compromising the privacy and security of their customers’ data.

Kulkarni says one of his prospects quipped that they don’t want open AI, they want closed AI. “I know that was kind of a tongue  n cheek comment, but they actually mean it,” he said. “They can’t send their data to a hosted model in the cloud where, despite some guarantee, there is really a risk of losing their content, losing their personally identifiable information. There would be a HIPAA violation if they did that.”

Stellar’s customers demand private cloud models that they can control, Kulkarni said. They want to know what data goes in, what data is shared, how the data is masked, and whether it’s synthetic data or real data. These observability requirements extend to responsible AI and traditional metrics of model drift, and bias detection.

Stellar has developed some frameworks that jumpstart the GenAI development process, but most engagements require additional tools, such as vector databases and tools like LlamaIndex and LangChain. The GenAI space is growing so fast that nobody has a complete end-to-end solution.

“I don’t think it’s realistic for anybody outside of Microsoft and Google to say we offer it all end to end,” Kulkarni said. “They offer some solutions, but even they haven’t covered all aspects of it yet.”

The Stellar team is enjoying working with clients in multiple fields, and it could possibly yield some shrink-wrapped tooling that the company could sell or open source in the future, Kulkarni said. But in the meantime, the company is just trying to keep up with pace of technological evolution and demand from clients.

While GenAI tooling will inevitably be better and more powerful in six months, it’s probably not worth the risk, Kulkarni said.

“I feel there’s a big risk for these enterprises if they wait and watch for too long because things are moving really fast,” he said. “You need to start experimenting and learning, or working with people who have done some experimentation and know some of these best practices to get ahead. You can’t wait too long.”

Related Items:

Pilot vs. Co-pilot: How Startups Will Reshape the Future of Work with AI

Happy Birthday, ChatGPT!

Will Mass Adoption of GenAI Lift Traditional AI?

Editor’s note: This article has been corrected. Stellar AI is based in Indianpolis, Indiana, not Sunnyvale, California. Datanami regrets the error.