Big Iron Warms Up to Big Data

Despite predictions of its demise, the mainframe keeps ticking. And thanks to its longevity, customers are finding the need to integrate mainframes with big data systems is growing. Syncsort, which develops big data solutions for mainframe customers, recently took a look at the state of big data in the mainframe community.

The key takeaway from the 19-page State of the Mainframe report that Syncsort released in early 2017 is this: 66% of 220 mainframe customers surveyed say the capability to do big data analytics for operational and/or security purposes is either “very important” or “somewhat important.”

This number shows that big data has definitely caught the attention of busy IT professionals in mainframe shops, which is no easy task, considering the business-critical nature of mainframe operations in Fortune 1000 companies.

While some analytics are performed on the mainframe, many customers opt to move the transactional data to other systems for analyses. This is not surprising, given how expensive MIPS can be.

The study found that 60% of IBM mainframe shops plan to move data off the platform for analytics, primarily using FTP but also via other common communication protocols. More than five out of 10 mainframe pros report using unnamed tools to perform batch analysis, while a smaller number use modern big data architectures like Hadoop, Spark, and Splunk.

Source: Syncsort State of the Mainframe study

However, mainframe shops don’t always have a good handle on this data movement, whether it’s done on the mainframe or off of it. The survey found that 62% of respondents don’t feel they can effectively track data in motion. “Organizations are looking for tools to monitor data movement across a variety of platforms and let them know what data is being moved, by whom, when and where,” the company says.

Despite the lack of visibility into data movement, mainframe shops are still eager to leverage other platforms for analysis.  About 49% of Syncsort’s survey respondents said it’s “desirable” to have access to log, SMF or other mainframe data to correlate it with other data using big data platforms like Hadoop and Splunk.

While big data is a topic in mainframe shops these days, as it is in most enterprises, it’s not necessarily the most important objective. According to Syncsort’s survey, “learning about big data analytics and tools” ranked as the 9th most important corporate objectives. The top two corporate objectives–meeting security and compliance requirements and meeting SLAs—got twice as many votes as big data analytics.

David Hodgson, general manager of Syncsort’s mainframe business, said that mainframe shops are increasingly eager to integrate data from the most critical transaction processing system with big data analytics platforms for improving business operations, detecting security breaches, and preparing for compliance audits.

“The mainframe is no longer the isolated black box it once was, and more organizations are realizing the immense benefits of performing analysis off the mainframe for real-time results, while investing in solutions that simplify data movement between platforms to bring together key enterprise-wide business information,” he says in a press release.

You can download a copy of Syncsort’s report here.

Related Items:

See EBCDIC Run on Hadoop and Spark

How a Web Analytics Firm Turbo-Charged Its Hadoop ETL

Syncsort Siphons Up Legacy Workloads for Amazon EMR