Hortonworks Preps for Coming IoT Device Storm

The masses may not be getting flying cars or personal robots anytime soon, but thanks to connected devices, the future of technology is definitely bright. From smart refrigerators and wireless BBQ sensors to connected turbines and semi-trucks, the Internet of Things (IoT) will have a huge impact on consumer and industrial tech. However, managing hundreds of billions of devices—not to mention the data they generate–will not be easy, which is why Hadoop distributor Hortonworks is taking pains to prepare for the coming device storm.

According to Intel (NASDAQ: INTC), the number of connected devices will explode in the near future, growing from about 15 billion devices in 2015 to more than 200 billion by 2010. Anybody who’s watching the rise of wearables like the Fitbit, the popularity of smart thermostats like Nest, or the presence of drone aircraft can tell you that this phenomenon is real and accelerating.

Getting these wireless IoT devices integrated with command and control systems will not be an easy task. While we now have enough Internet addresses to go around, there’s a lot of other messy details that need to be worked out to ensure that devices aren’t stepping on each other’s toes, that they can’t be co-opted by others, that the data is safe and secure. Right now, it’s an ad hoc, Wild West IoT world, but that simply won’t scale.

Hortonworks (NASDQ: HDP) announced a month ago that it was teaming up with a company called Neustar (NYSE: NSR) to help drive some standards into the IoT device connectivity and application development racket. Based in Virginia, Neustar does a lot of work behind the scenes to make sure the Internet and phone traffic is routed securely. You may not have heard of it, but the $1-billion company runs one of the largest Domain Name Service (DNS) registries in the world.

The companies are working to develop a software platform that handles much of the underlying plumbing needed to build secure and scalable IoT applications. Today, that work is done as expensive one-off projects, says Hank Skorny, the senior vice president of IoT at Neustar.

“It’s almost like back to the days when you wanted to print something out you had to write custom code to talk directly to the printer,” says Skorny, who previously ran Intel’s IoT business. “Today that’s how it is in the IoT world. If a person wants to add a new device or a software solution, you have a systems integrator who walks in the door and says, ‘Write me a check for $2 million to $10 million.'”

At last count, there were 24 different operating systems used in the IoT, Skorny says. IoT devices commonly use processors from Intel, Arm, Qualcomm, or Wind River. Raspberry Pi anyone? And don’t even mention communication protocols. They are all over the map. Your wireless barbeque sensor might use Bluetooth while your smart LED bulb talks ZigBee. What’s an aspiring IoT entrepreneur to do?

“There’s no standardization,” Skorny continues. “There’s no easy way to do it. Everything is custom. You can’t upgrade the systems.” And there’s no standard way to go from a single prototype device to the nth device.

There’s no doubt that IoT growth has been organic. Like weeds in spring, the patchwork of different hardware, OSes, and communication protocols has grown unabated. This is to be expected when technological progress meets capitalist creativity.

But in the long run, the diversity of technologies and protocols and policies will hamper the growth of the IoT and the analytic applications that are slated to grow out of it.

“It has to be a more horizontal platform or else developers are going to be driven nuts,” Skorny says. “You’re going to have a whole bunch of conflicting protocols, and you’ll have conflicting policies, which is a big problem.”

Neustar and Hortonworks are collaborating on a suite of software that aims to be that horizontal platform for building and scaling IoT applications in the cloud. The product would work with Hortonworks DataFlow product, the real-time streaming analytics and IoT platform that it announced in August following the acquisition of Onyara, as well as Apache Atlas, the data governance tool for Hadoop. The plan calls for the new software to be developed in the open and ultimately contributed to the Apache Software Foundation.

Several groups are developing software platforms that allow users to build IoT apps, but they’re doing so from a narrow, vertical-industry perspective. “To be able write and interact with all these devices using standards and open source [platforms] like Hadoop is going to be critical to taking the IoT out of the vertical world that it exists in today and into a truly horizontal world where one can write any application for any device,” Skorny says.

The application, which doesn’t have a name yet, will largely sit above level 5 on the OSI stack. It’s about more than just communicating with an IP address, Skorny says. “You need to figure out what device it is. Is it a temperature sensor? If it is, what protocols does it use? What APIs does it accept? What version is it? How can I securely authenticate it? And can I securely authenticate it in a multi-factor way repeatedly through a session to make sure the device doesn’t go rogue on me.”

The Internet’s security model is largely based on an organization chart, and was designed for humans, Skorny says. But that doesn’t approach simply doesn’t translate into the IoT world, where there are 200 devices per person.

“The key missing element that I saw in working with all of our partners is there’s no master repository or registry of IoT devices,” he said. Such a repository would deliver universal discovery and plug-and-play capabilities, along with secure authentication.

It seems almost predetermined today that devices in the future will connect to networks, pushing and pulling data, making us happier and healthier. The analog world is but a distant memory. But the IoT’s future has not yet been written, and things could gravitate further toward chaos or coalesce around a few basic principles. Hortonworks and Neustar have identified the need and are working to standardize things. Only time will tell if it’s the path we ultimately follow.

Related Items

Hortonworks Boosts Streaming Analytics, IoT Plays with NiFi Deal

The Data of Things: How Edge Analytics and IoT Go Hand In Hand

Hacking Your Way Onto the IoT