Solid-State Drive Makers Eye Big Data Markets

Flash and other forms of solid-state memory are increasingly being leveraged as high-end storage solutions. As hard-disk drives in servers failed to keep pace with data loads, generating performance bottlenecks, solid-state drives are finding more big data applications as customers ingest billions of data points on a daily basis.
In a case study sponsored by solid-state drive maker Micron Technology Inc. of Boise, Idaho, an advertising firm that attempts top track users Web browsing, app usage and even physical visits to retailers leveraged SSD technology for data analytics. The goal was to sift through mountains of raw customer data to determine customers’ affinity with certain brands.

The advertiser’s technology challenge was to find a faster, more agile solution to analyzing data, then connecting customers to brands through targeted advertising.

The sheer scale of the big data challenge is staggering. From the beginning of human history to 2003, the Smithsonian Institution estimates that 5 billion gigabytes of data were generated. Today, the same amount of data is being generating every 10 minutes!

The ad firm had been using SAS hard-drive arrays with plans to scale them up as databases bulged. But it eventually hit performance bottlenecks as the amount of customer data soared.

After deciding they had reached the end of the line, the agency looked for an affordable solution without having to purchase hundreds of additional servers. It also needed extremely low latency and high throughput since its matching algorithms, the ad firm’s family jewels, require fast processing of vast amounts of data about online advertising exchanges.

Given that Micron sponsored this case study, the advertising customer naturally went with their SSD. However, Micron makes several types of solid-state drives, so the ad firm had to decided which was best suited to its operations. It decided that a higher-end application accelerator was not required.

Instead, the ad firm concluded that it needed a single-boot drive. Hence, it went with a lower-end Micron SSD dubbed P400e.

For applications that gather larger amounts of customer data on a daily basis, the ad firm went with higher-endurance mainstream SATA SSDs.

The lesson was that one type of SSD doesn’t solve all problems. And given the expense of SSDs compared to standard hard drives, customers must identify the solid-state drive that matches its performance requirements and application suite.

Micron and other solid-state drive and flash memory makers are slowly making inroads into the big data market, although flash arrays have so far made a bigger splash in the enterprise storage market via well-heeled startups like Pure Storage. The challenge for memory and drive makers will be convincing customers that the performance increases they provide are worth the investment.

Moore’s Law and higher volumes will also help bring down the unit costs for emerging SSDs, which could open up more opportunities in a big data market where agility is becoming a differentiator.