Investment in Machine Learning Keeps Growing, DataRobot Finds

(Blue Planet Studio/Shutterstock)

After years of double-digit growth in machine learning investments, nearly 90% of companies grew their machine learning budgets again from 2020 to 2021, according to a survey from DataRobot, which also documented the top struggles that companies have with machine learning.

For DataRobot’s report, titled “5 Latest Trends in Enterprise Machine Learning,” the AI cloud vendor surveyed about 400 companies about their AI and ML strategies. The survey showed that 86% of the companies surveyed planned to increase their spending on ML, compared ot 10% who planned no increase and just 4% who planned to spend less.

While the number of companies planning to increase their 2021 spending on ML by 1% to 25% relative to 2020 declined compared to the previous two years, DataRobot spotted significant increase in companies planning to increase their spending by 26% or more.

Where is all this money going? While some of it’s going to better tools and technology–such as software from DataRobot and other companies in the data space–a big chunk of it’s going to personnel. Specifically, it’s going to data scientists and data engineers.

Incredibly, DataRobot’s survey found that 57% of survey respondents have 50 or more data scientists on staff as of the middle of 2021, with 25% of those firms (or 50 out of the 400 surveyed) employing 100 or more data scientists. Considering the difficulty that companies have reported in recruiting and keeping data scientists, that number is remarkable (although it could be skewed if DataRobot surveyed predominantly very large companies).

What is even more amazing is that the figure is actually down slightly from the beginning of the year, when DataRobot’s survey showed 58% of respondents had 50 or more data scientists on staff. Compared to 2020, companies have hired many additional data scientists, the DataRobot survey shows. They are also hiring machine learning engineers, as the company found the ratio between data scientists and machine learning engineers to be roughly one to one.

The vast majority of companies surveyed by DataRobot are increasing their ML spending (Source: DataRobot survey)

When it comes to top challenges in the data science and machine learning realm, there is plenty to talk about. Ninety percent of survey respondents say they “struggle with complex infrastructure or workload needs,” which is the top concern. The survey found 88% of survey respondents struggle with integration and compatibility of AI/ML technologies, while 86% struggle with the frequent updates required for data science tooling.

The widespread struggles with tooling updates is telling, says Michael Azoff, a consulting analyst with GigaOm.

“The most common alternative to a single integrated AI/ML lifecycle solution is a build-your-own solution, frequently built from open-source components,” Azoff says in the DataRobot report. “Organizations that take the BYO route put in a lot of effort up front to create an integrated solution, but often neglect to account for the maintenance and upgrades as the various components in their tool chain release security patches and new editions (and all at different times).”

When it comes to model deployment and management, 63% of the companies DataRobot surveyed say they’re using third-party tools, such as those offered by DataRobot and Algorithmia, which DataRobot acquired earlier this year. That means 37% are essentially rolling their own in this department.

The DataRobot survey also asked survey respondents about the Kubernetes distribution they use. The results are quite varied, with the Google Kubernetes Engine from Google Cloud, AWS’s Amazon EKS, and the Azure Kubernetes Service from Microsoft Azure each owning 20% to 30% of the market, followed by Red Hat OpenShift with 11, and 7% each for Minikube and the original open source distribution of Kubernetes maintained by the Cloud Native Computing Foundation

Related Items:

DataRobot Nabs $300M, Algorithmia

DataRobot Refreshes AI Platform, Nabs Zepl

DataRobot Eyes IPO After Another VC Haul