Half the World’s Data to Touch Hadoop by 2015?

It’s no secret that Hortonworks has placed its bets on Apache Hadoop, but the company has extended its view of the long-term success of the open source goldmine.

According to Shaun Connolly, VP of the company’s corporate strategy, Hadoop with touch or process 50 percent of the world’s data by 2015, a staggering figure, especially if the multi-zettabyte prediction for world data volumes is true for that year.

Connolly says that the fundamental drivers leading the interest in Hadoop and the adoption of Hadoop really are rooted in estimates like those from analyst group IDC, which estimates that the amount of data that enterprise data centers will be processing will grow by 50x.

Beyond data volume, however, is an equally important issue. Connolly says that if you look at data flow and what enterprise data centers, there is a gap in terms of how data is being handled. He states that 80% of that data flowing through enterprise data centers will be required to touch in some form or fashion–users may not store it, but it will flow through the business. 

What this means, says Connolly, is that volume, the velocity, and the variety of data flowing through enterprises are key elements. He notes that if one is to look at traditional key application architectures today, they’re really not set up for that challenge, and it’s a challenge that requires a fundamental rethink of architectures overall. This is where Hadoop comes in…

The Hortonworks business lead claims that Hadoop can address these weaknesses because it  was purpose-built to address those three V’s of big data (volume and the variety and velocity). It can store unstructured data, semi-structured data very well across commodity hardware, but says Connolly, it’s able to do it in an economically viable way, which makes it a standout technology option.

Hortonworks says that to stay ahead of the curve and prepare the Hadoop era, broad availability to technologies is step one—a fact that they say is covered by their commitment to open source software.

The second element of the equation is creating a broad and vibrant ecosystem in and around that, and enabling that ecosystem.  As Connolly says, “It sort of gives us hand and glove with the open source technology, providing open APIs around that platform, but really focusing at Hortonworks as a business. When enabling sort of the key vendors and solution providers and platform vendors up and down the data stack, to be able to integrate their solutions that may already be within the enterprise today. Integrate them very tightly with this next generation data platform, so they’re able to offer more value to their existing enterprise customers.”

Related Stories

Six Big Name Schools with Big Data Programs

Analyst Group Crowns Hadoop Distro King

Hortonworks Hocks Hadoop Upgrade