ABBA Seeks to Top Charts with Flash

ABBA is the source of more than just upbeat disco grooves. These days, the band’s acronym is being revived to back an EMC project that hopes to use Flash technology to spin big data into fast data.

Although we too agree that there’s just not enough gold lamé in data-intensive computing R&D, sadly, the Swedish pop team of Agnetha, Björn, Benny, and Anni-Frid has nothing to do with this research. EMC’s rendition of ABBA stands for Active Burst Buffer Appliance, which is behind their effort to create a fast data big data solution.

Dr. Patricia Florissi, EMC’s Global Chief Technology Officer detailed the concept of “fast data” for large, complex data sets recently, noting that the point of high performance architecture is to split up problems through parallelization such that each node is working on a tiny bit of the problem.

As Florissi explains using the example of electrons smashing together in a one-meter box, “Even in 2012, modeling a single cubic meter may take 20 years. One way to reduce computation time is through parallelization. One way to parallelize this computation is to break it up into a million cubic centimeters just like a Rubik’s cube is divided into smaller cubes. You can then assign each cubic millimeter to a node and have all the nodes run at the same time, decreasing the compute time by many orders of magnitude.”

Naturally, as the size and complexity of the problem grow, so does the necessary architecture and number of nodes. Also, it is typically not good enough for these nodes to be acting independently in parallel. Since each node is given a certain area of space in the one cubic meter electron-filled box example, in this case a cubic millimeter, the nodes are required to interact with each other when electrons near their borders and, by analogy, other nodes. These nodes are said to be “tightly coupled.”

“High performance architectures,” says Florissi “commonly use thousands of commodity compute nodes, and any of these nodes can fail at any time. These computationally intensive applications are very tightly coupled, requiring every node to work in tandem like a team. Because of the tightly coupling, when a single node fails, the entire computation fails and needs to start again.”

When a node fails in a tightly coupled calculation, the entire calculation fails and needs to be restarted. In order to reduce the catastrophic nature of single node failures, referral checkpoints need to be set up at regular intervals. This is done by introducing storage nodes, to store the data for each successive checkpoint so it can both record the calculation’s progress and provide a backup when the inevitable node failure happens.

But these are all characteristics of high performance architecture, explained by Florissi in the previous video. The problem with this architecture, or at least a problem that prevents it calculations from happening in a desired short amount of time, what Florissi calls “Fast data big data,” is that all of the compute nodes must go idle when writing their information to the storage nodes.

While data storage, including big data storage, is measured in bytes to petabytes, data speed is measured in flops to petaflops, or floating point operations per second. According to Florissi, Los Alamos can handle teraflops to petaflops. That pales in comparison to India Space Research Organization’s aspirations. By the year 2017, ISRO will hope to have built a supercomputer which can handle 132.8 exabytes, storing 3-5 petabytes in less than five minutes.

The problem that EMC wishes to solve is two-fold: decreasing compute nodes’ idle time when transferring data to the storage nodes for checkpoints and decreasing the amount of time it takes for storage nodes to feed information back to the compute nodes when they fail. Typical big data or fast data solutions will not cut it for Florissi. “Fast data big data renders useless existing alternative approaches to deal with the limitations of the underlying environment.”

For Florissi, that is where Flash technology comes in. “Flash technologies come in with a throughput rate of a terabyte per second as opposed to 170 megabytes per disk drive.”

EMC is working on, through ABBA, using Flash appliances as a sort of a high-speed buffer that is compatible with both the compute and storage nodes. “This design brings together two crucial elements in increasing the performance of checkpoints: one, a faster network to unload the data off the compute nodes; and two, low latency, non-volatile storage to store the data.”

Put simply (something of which the video does a great job), the Flash appliances connect to the nodes at higher speeds. Further, if one secures enough Flash appliances (two or more per compute node), some of the Flash nodes could write the upcoming checkpoint while the others are busy writing the previous ones. This would allow the compute nodes to run continuously.

Hypothetically, the checkpoints could be running continuously as well if they have no effect on compute node downtime. This would produce a significantly higher amount of data for the storage nodes but it would also provide continuous insight into the calculation. Continuous saving also minimizes the calculation loss when a compute node fails.

Other potential solutions, such as installing Flash on each compute node or each storage node, are either too labor intensive (as noted above, there existed a million compute nodes to model one cubic meter of electrons) or do not address the problem, hastening the relationship between the storage and compute nodes, that EMC faces according to Florissi.

For example, if Flash is installed on each individual compute node and a single compute node fails, that storage becomes trapped. This can be allayed by pairing each compute node with a Flash buddy, but that would require double the appliances, at which point one might as well set up the double ABBA barrier.

HPC architecture as it is generally includes nodes which are designated for input and output. However, Florissi sees that evolving to fit ABBA. “In existing HPC architecture, the Flash can connect to the existing I/O nodes, minimizing architectural changes. In the future, however, Flash appliances can become the I/O nodes themselves. Now, the compute nodes save the checkpoints in the I/O nodes, which happen to be Flash appliances themselves…Because the Flash appliances are active appliances which can handle the data transfer without consuming compute node cycles.”

ABBA the musical group topped the UK charts with their Greatest Hits album in 1976. A little over 40 years later, ISRO will hope to top the fast data supercomputer charts using some form of EMC’s ABBA.

And now, since the repeated mention of ABBA in this article has doubtlessly left you with “Dancing Queen” blazing through your brain, we will leave you to indulge. Next week you bring an article on MADONNA, which stands for Massive Aware Data Ontology (with) Network-Named Antennae.

On an entirely unrelated note, we’re pretty sure the blonde one is an ancient vampire.