AI Infrastructure Gets a Stack

In an effort to create a standard set of tools that would help data science teams collaborate on AI development, an infrastructure initiative launched this week will promote a unified stack for developing and scaling machine learning models.

The AI Infrastructure Alliance said this week it will initially focus on creating Canonical Stack for AI envisioned as a development platform for machine learning models destined for enterprise applications. As with previous hardware and software stacks, the machine learning initiative seeks to forge an AI development infrastructure that would free developers to address more complex problems.

As machine learning models move to the edge, the alliance said it would create a single platform that integrates existing AI technologies into a common framework that would accelerate and improve MLOps and edge applications.

Establishing a so-called canonical AI stack for machine learning and MLOps would include developing best practices and architectures used to scale machine learning models in edge and other applications. The alliance also pledged to promote open algorithms, data sets, frameworks, libraries, models and tooling along with developing standards for APIs used to share data and metadata among machine learning applications.

It will also promote technologies used to anonymize data sets to protect privacy along with homomorphic encryption to strengthen data security.  Homomorphic encryption permits calculations on data without decryption.

“Despite a massive surge of partial solutions, no single tool exists that lets teams leverage the true power and potential of AI,” said Dan Jefferies, the AI alliance’s director and chief technical evangelist at Pachyderm. “The AI Infrastructure Alliance will help create clarity in this confusing space by building a cohesive framework and bringing together leaders and innovators to help set the standard for how data science teams build models now, and into the future.”

Among the alliance’s 25 charter members are MLOps vendors such as Algorithmia and Pachyderm along with deep learning specialist Allegro.

According to the group’s mission statement, AI and machine learning software currently “sits squarely in the early adopter phase of the technology adoption curve. As competitors and researchers work to create solutions to the unique problems of data science, it results in a massive proliferation of tools that creates tremendous confusion in the marketplace.

“Enterprise organizations everywhere are struggling to stitch together dozens of different tools to create a complete AI/ML platform,” the AI alliance noted.

Hence, the alliance will seek to forge a standard AI framework capable of running any application, model or development tool.

Recent items:

In Search of a Common Deep Learning Stack

Kubernetes Gets an Automated ML Workflow