Quantum’s New Reference Architecture Tackles Autonomous Driving

San Jose-based Quantum Corporation aims to operate at the cutting edge of data management – and that edge, of course, is moving rapidly towards AI and machine learning. Now, the company is announcing a new reference architecture targeted at supporting autonomous driving systems and industrial development of AI/ML applications.

Notoriously, autonomous vehicle training requires gargantuan amounts of high-definition video, LiDAR data, and sensor data captured by the onboard technology used to equip the vehicles. (Tesla, recently, has been detailing its efforts to create new supercomputer architectures to handle the sheer difficulty of managing and processing the data used in autonomous vehicle training.) Plamen Minev, technical director for AI and cloud at Quantum, called autonomous vehicle training one of the “most extensive and highly visible AI/ML use cases”.

“Each vehicle collects terabytes of data per hour, which needs to be stored reliably and offloaded to a datacenter at a later time,” Minev wrote. “The data collected at the datacenter quickly grows to tens and even hundreds of petabytes.”

“Although still relatively nascent, organizations developing autonomous vehicles are at a crossroads,” added Jamie Lerner, president and CEO of Quantum. “The volume of data being captured is increasing exponentially, presenting an urgent need for speed, capacity and cost-efficiency in the data management lifecycle.”

Quantum claims that its new architecture constitutes the “only single-vendor solution on the market that covers all phases of the [autonomous vehicle] development process, including in-vehicle data collection, data preparation, ML model training, system simulation, [hardware-in-the-loop] testing, long-term storage, and archiving.”

The architecture includes the Quantum R6000 storage appliance, which is stored in the trunk of a test car and is capable of collecting more than 10GB per second. The R6000, in turn, offloads the data to a datacenter, where it can be stored in Quantum’s StorNext File System. There, Quantum says, the active data (between one and six weeks old) is made available for analysis and HPC applications, while the inactive data (more than six weeks old) is moved to protected cloud, tape, or objective storage archives.

Quantum’s reference architecture for autonomous vehicles. Image courtesy of Quantum.

“These are challenges that Quantum has been solving for over 40 years in other sectors,” said Graham Cousens, practice lead for advanced driver-assistance systems and autonomous driving solutions for Quantum. “[Driven] by the powerful StorNext File System and our ultra-fast automotive and mil-spec R6000 in-vehicle data storage device – this new reference architecture is set to streamline and power the future of autonomous vehicles development.”