Finding Your Way in the New Data Economy

(Adisa/Shutterstock.com)

In big data analytics today, the goal is to capture, cleanse, and analyze as much data as possible. The leading practitioners are blending dozens of external sources with their own data sets, maybe more. But in the emerging data economy, you’ll not only be tapping into a pool of available data that’s exponentially larger, but you’ll be contributing your data to it as well.

There is so much activity underway in the data and analytic space at the moment that it is impossible to keep track of it all. It seems at times we’re in the midst of several revolutions that are simultaneously playing themselves out, and each of these is impacting the world of data analytics in its own way.

First, we’re witnessing the rise of parallel computing architectures, such as Hadoop, Mesos, Spark, NoSQL, and NewSQL databases. The worlds of machine learning and deep learning are progressing rapidly, resulting in new advances in cognitive computing and artificial intelligence. And the World Wide Web continues its relentless advance outward, hooking billions of devices into a universal data grid, all hooked in via APIs, called the Internet of Things (IoT).

One of the side effects of these revolutions is the rise of a new data economy, where organizations buy and sell data as commodity. Companies that had previously discarded small pieces of data “exhaust” that they didn’t think were valuable are finding that somebody, somewhere, can make use of that data, and they’re willing to pay for it.

Mapping technology becomes more prominent in the emerging data economy

“The world is now this grocery store of data sources. You can walk in and purchase or rent sources,” says Jack Phillips, the CEO of the International Institute for Analytics. “There’s probably some vendor or company capturing data in the area you work in. If it’s in healthcare and you look at the average hospital or the average ER room, it is an IoT laboratory. There’s no shortage of measuring. The shortage is imagining what if we were to pair this and this and these data to answer our question.”

The new data economy is completely transforming how we think about business intelligence, resulting in “runaway acceleration” in innovation, according to Ali Tore, the chief product officer at ClearStory Data.

“We live today in a unique data landscape context,” Tore wrote on the ClearStory blog. “We also live in a unique time of business competition. Both are in a state we’ve never seen or experienced before. Add to that fundamental technology advances that have occurred in cloud, user interfaces, and data processing, and together we are now seeing a kind of runaway acceleration in BI innovation that surpasses previous decades.”

The current wave of transformation will apply to everything, Tore wrote, “from the way disparate data is accessed, how it’s prepped, how it’s blended, how fast it happens, how easy it is, and how it’s freely explored and analyzed.”

Data has no mass, energy, nor momentum. But it still possesses intrinsic value, and behaves like a capital good. In fact it’s leading the world’s biggest companies to spend billions to get more of it. Geographic data is so valuable now that Audi, BMW, and Daimler just gave Nokia $2.8 billion to buy HERE, the digital mapping company that was originally called Naviteq.

(Turgaygundogdu/Shutterstock.com)

“Why is GE (NYSE: GE) spending multi billions of dollars on a research facility to manage data?” asks Ramon Chen, the chief marketing officer for Reltio, which provides a cloud data management platform that automatically sources, prepares, and analyzes data for clients across all industries, with a focus on healthcare and life sciences. “Yes, they want to improve efficiencies of their business and consolidate the cost of IT. But the ulterior motive here is that data is valuable. They can sell that data for a whole heap of money.”

You can see reflections of the emerging data economy in IBM‘s (NYSE: IBM) decision last October to spend a reported $2 billion to buy the Weather Company. Instead of renting the weather forecasts from the Weather Company, as it had done in the past, Big Blue decided to buy the whole shebang and get into the weather data business itself.

The forecast gets clearer when you consider IBM’s moves in the emerging IoT business. You can see how a trucking company, for example, might combine a weather data feed from IBM with data from its dispatching system to get a better picture–via the Jeopardy-beating Watson supercomputer, of course–of how a storm will impact deliveries. IBM could package this all up as a service, and sell it as part of an a la carte package.

In the emerging data economy, the organization with the best local data wins, says Lukas Biewald, the CEO of data cleansing firm CrowdFlower. “As stuff gets more local, it gets harder,” he says. “If you just want to find all the companies in the New York Stock Exchange, it’s not that hard. Everybody has that. But I don’t think anybody has a complete list of restaurants in San Francisco. It changes every day!

Real-time weather data becomes a differentiator in the emerging data economy.

Don’t be surprised to see some companies from the old economy—the ones who operated by making, moving, and maintaining physical things, like the ones that IBM still sells mainframes to—give up and go under in today’s uber-competitive business climate. In many cases, the acquirer won’t give a hoot about their factories, warehouses, and fleets of trucks. Instead, they’ll just be after the data.

“If you’ve been around a long time, you’ve probably collected data that nobody else has access to,” Biewald says. “As people want to do new kinds of machine learning models, you need data sets to have a competitive advantage. Folks like IBM and Oracle (NYSE: ORCL) and Google (NASDAQ: GOOG) have the machine learning algorithms and are now looking to see where they can get proprietary data that they don’t have, to get competitive advantage.”

The alternative, of course, is for the old-economy companies to retrofit their business models to fit the new data-economy realities. IBM built and ran the business machines that powered the world’s biggest companies of the last century, and is in the midst of re-tooling itself to build and run the next generation of business machines.

Capitalistic cultures have always embraced competition, but increasingly, the competition is about who can best navigate the tumultuous waters of the emerging data economy. If you can’t navigate the new data economy, it doesn’t matter if you’re good at your job—like running a trucking company or making food that people want to eat. Being good at computers and manipulating data is suddenly important, no matter what industry you’re in.

Call it the Uber Effect, after the super-successful taxi company (valuation: $65 billion) that doesn’t own a single car. Even old-economy companies that are proficient at 20th century business intelligence (i.e. data warehousing, SQL, OLAP, etc.) techniques are in the cross-hairs of data-driven disruption, says Ashish Thusoo, the CEO of Hadoop as a service firm Qubole and a co-creator of Apache Hive.

“It does count, to some degree, but in industries where technology and BI have never been used before, the startups have tremendous advantages, for sure,” Thusoo says. “Many of those older industries are operating on very antiquated methods and they don’t have the DNA…to really use those [big data] techniques.”

The advantage today lies with the upstarts and the startups in many industries. Consumer-focused industries are being disrupted first, but even the slower-moving business-facing industries will be disrupted by the data economy.

Related Items:

Is 2016 the Beginning of the End for Big Data?

Is Your Organization Making the Best Use of Its Big Data?

Automakers Bet Big on Data and Analytics