Why Event Meshes Should Be On Your IoT Radar

We are instrumenting everything around us at a frenetic pace with the hope of using all that sensor data to improve our situational awareness and decision-making capability. But how do we plumb all the systems together in a simple and cohesive manner? That, in a nutshell, is the purpose of an event mesh, an emerging product category you’re likely to hear more of in the future.

Shawn McAllister, the CTO of Solace, is an expert on event meshes and recently briefed Datanami on the emerging field. As McAllister explains it, event meshes help to virtualize and simplify access to event data in the same way that service meshes help to virtualize and simplify access to IT services.

“An event mesh is another layer in your network that delivers events to you, no matter where you happen to be,” McAllister says. “It gives you the ability to take applications that may be deployed in your legacy environment, in your private cloud, and multiple public cloud environments, and have all those applications are able to interact and to receive whatever events they’re interested in.”

The event mesh starts with event brokers that are deployed close to the item being monitored, McAllister continues. Those event brokers are typically installed next to your legacy applications, which can now emit events into the event mesh, which routes them to consuming applications on the other side of the network.

It’s all about shrinking the physical world and making it more connected. Remote field locations, such as gas stations or grocery stores, can now be the source of an event that triggers an action half a world away. That means events, like purchacing gasoline, could trigger a machine learning system running in the Google Cloud to give the customer an offer, via the pump’s video screen, to get a discount on energy drinks purchased in the gas station’s mini-mart.

Event Intelligence

Gartner is a big backer of the importance of events (and thus event data) in the enterprise. In 2018, the company said that 80% of new business ecosystems will require support for event processing. And by 2022, Gartner says that event-sourced situational awareness will be a “required characteristic” for new business solutions.

Gartner backs the event mesh concept in its vision for platform architectures (image courtesy Solace)

Solace’s software allows companies to implement event meshes in their IT architecture. Its offerings include a distributed publish and subscribe (PubSub) messaging bus that can push events over the network in a fast and reliable manner. The company also provides event broker software for seeding the events into the network.

“What you do is deploy event brokers in each location, each close to the applications,” McAllister says. “So event brokers, and the event mesh they form, take responsibility for persistent or lossless delivery of events, in order, with security, with robustness, with performance, distributed to these many places.”

Solace has quietly built a business around the event mesh concept. The company, which is headquartered in Ottawa, Ontario and has 15 offices around the world, employs more than 250 people and owns 24 patents related to event meshes. Its software is used with event meshes used by large banks, automakers, telecommunications firms, and air traffic control stations around the world.

“We have customers where we manage their elevators, we connect their cars, we connect their factories,” McAllister says. “So these things are becoming very intelligent and you want to connect them into your event-driven enterprise. So as you sell Big Macs, you put some promotion online. You want to do real-time inventory and do real-time offers at gas pumps and things like that based on me swiping my loyalty card.”

More Than Stream Processing

Solace attended the recent Kafka Summit event in San Francisco. While there is some overlap with Apache Kafka, which also implements a distributed PubSub messaging bus, there are key differences in the capabilities that Solace and Kafka offer, McAllister says.

“From the Kafka point of view, it’s very tailored to streaming and stream processing,” he says. “Yes, it does PubSub. But if you look at other requirements in the enterprise, there are lots of situations where having a Kafka client library kind of just doesn’t cut it.”

Event meshes are formed from a network of event brokers (image courtesy Solace)

For example, if a customer wants to the ability to not only receive data from an IOT device but also have the ability to control it, they would need to take a different approach, such as using MQTT protocol, he says.

“Do companies like Confluent have an MQTT adapter?  Yes,” McAllister continues. “The thing is, that is used for data aggregation into the Kafka cluster. For reasons of topic scalability and real-time [notifications], the ability of that application to do command control back to devices is not at all well suited for going through Kafka to an MQTT adapter out to that device.”

There are multiple ways to string event-emitting applications together, and each approach brings its own set of advantages and disadvantages. The advantage that event meshes bring is reducing the technological complexity for the developer when connecting existing applications with event-emitting devices on the IoT.

For example, say a developer needs to write a program that does something whenever somebody open a new account. How would the developer go about doing that? With an event mesh, a lot of the underlying architecture is already built for them.

“What I would do is I would find the event that’s emitted when somebody opens an account. I would subscribe to that event, and then I would be notified every time that happens,” McAllister says. “If my event processing is infrequent, I can do it as a function. It could be Lambda in AWS or Kubernetes, for example.”

Building on an event mesh framework protects the developer from much of the underlying complexity and dependencies that would otherwise be required to build an integrated event-driven architecture, McAllister says.

“The application that opens the customer account doesn’t know, and doesn’t need to know, who else cares about the event,” he says. “So he can just announce to the event mesh, ‘Hey there was an account open.’ And now from that point on, a whole bunch of applications can be built consuming the account open event or new customer event, and they can do a whole bunch of things without having to go back and change the application that opens the account.”

Plumbing for Events

With the event mesh architecture already established, developers can build more complex event-driven applications faster than if they had to instrument the applications themselves.

“What we see is that your legacy applications, whether they run on a mainframe or system of record like SAP or Amdocs, or legacy messaging applications — these run your business. You can’t throw them away. You need to continue to leverage them,” he says. “But if you can make the things happen in those applications available across lines of business, in real time, to the various places that you are building applications, now you can become very agile with respect to ‘the business wants me to do X.’ Hey no problem. I’ll just subscribe to these events. I’ll get notified when something happens. Then I run my logic.”

Event meshes provide an intelligent routing system for event data (image courtesy Solace)

Speaking of SAP, the German software giant has OEMed Solace into its own offerings. “SAP’s enterprise messaging in the cloud uses Solace technology inside,” McAllister says. “And S/4 HANA is natively event-enabled and pushes events into a Solace enterprise messaging layer.” Ariba is also a big user of the Solace tech, he says.

McAllister worked in network routing for many years, while Heinz Schaffner, the principal technologist at Solace, worked at TIBCO for many years. This gives the technologists a different perspective on how to connect data in the emerging IoT world.

“We’re solving this problem of real-time reliable event distribution to many different places and being able to connect into SaaS applications, microservice applications, etc., at the edges of the mesh, and to do this reliable with performance and security,” McAllister says. “You don’t want applications to have to do this. You don’t want bespoke integration. You don’t want pair-wise integration. You need security built into your infrastructure. I very much view this as a layer that people need to have in their enterprise for many of the same reasons that service mesh is an accepted layer in your Kubernetes environment.”

The IoT, the arrival of 5G mobile broadband, and advances in edge computing are moving the data center and the real world closer together. Enterprises that find ways to augment existing systems in the data center with event-emitting devices in the field will find a competitive advantage. An event mesh may not be the only way to accomplish this, but it looks like it will be a contender in the years to come.

Related Items:

Harvesting IoT Insights, from Edge to Core

Why Data Scientists Should Consider Adding ‘IoT Expert’ to Their List of Skills

Built to Last: Laying a Framework for IoT with Enterprise Architecture