What 2017 Will Bring: 10 More Big Data Predictions

(Tortoon/Shutterstock)

2017 is finally here. But what will the New Year bring? We leave that up to our crack team of industry prognosticators, who have taken it upon themselves to issue a range of predictions for the field they know best: big data analytics.

We start out this second round of predictions (you can find the first round here) with everybody’s favorite big data platform: Hadoop. The distributed computing system has had a rough go of it lately, as people question whether the complexity of implementing Hadoop outweighs the big data benefits it brings. That question will quietly go away in 2017 as companies begin using it as a platform for their own software, predicts Andrew Brust, a Senior Director at Datameer.

“Hadoop (YARN and HDFS) is a super-powerful distributed computing platform,” Brust says. “[In 2017], an increasing number of companies will start using it as infrastructure for their own offerings, rather than as a directly exposed platform.  These companies will treat Hadoop more like an operating system and less like an application, abstracting away Hadoop’s complexity and turning its cluster-based deployment into a mere implementation detail.  Customers will get the power of the Hadoop platform and of the execution engines designed for it, without needing expertise in the platform or those engines themselves.”

But not everybody is so bullish on the yellow elephant’s future. In particular, the big data integration software firm SnapLogic sees problems continuing with Hadoop, but also possible solutions.

What will 2017 bring for the cuddly yellow distributed computing system?

“Fifty percent of Hadoop deployments have failed,” the company asserts. “While it’s commanded the lion’s-share of attention, it’s suffered from product overload. Because new projects are added every month and the nature of the data in the Hadoop cluster is ever-growing, it’s created a complex, multidimensional environment that’s difficult to maintain in production. To actually make Hadoop work beyond a test environment, enterprises will shift it to the cloud in 2017, and abstract storage from compute. This enables enterprises to select the tools they want to use (Spark, Flink or others) instead of being forced to carry excessive Hadoop baggage with them.”

According to Ketan Karkhanis, SVP & GM of Analytics Cloud at Salesforce, 2017 will be the year big data analytics finally delivers the value that executives have failed to get from business intelligence over many years.

“For so long, business leaders have been fundamentally unhappy with the promise of BI,” Karkhanis says. “They expected analytics to solve their problems, and now in the last year business leaders have woken up to the fact that pretty charts and graphs alone do not create real value. 2017 is the year that analytics will become a competitive differentiator, as c-level executives understand the potential data holds for each and every employee to work smarter.”

The battle between CPUs and GPUs will continue to play out, as backers of both processor technologies vie to absorb the most big data workloads. Count Todd Mostak, the founder and CEO of MapD Technologies, firmly in the GPU camp, especially as the IoT continues to generate machine data in heaping portions.

“CTOs and CIOs will face the reality that their CPU-era analytics solutions don’t scale elegantly or economically and are poorly suited to the new requirement of both streamed data and near-historical analysis (looking back 30, 60, 90 days),” Mostak predicts. “This will precipitate broad deployment of GPU-powered analytic solutions in these infrastructure-oriented industries. Telco will be the first industry to adopt this widely.”

AI-powered bots are poised to take over many digital tasks for people.

People are afraid that artificial intelligence will eventually take over many of the jobs that humans currently do. But a more benign reality is forecast by Automation Anywhere CEO Mihir Shukla, who caught of glimpse of 2017 as the “Year of the Future Workforce.”

“Seventy-five percent of Fortune 1000 companies will have humans working side by side with robots, and will have dipped their toes in creating a digital workforce by the end of 2017,” he says. What’s more, this hybrid digital-human workforce will generate “unprecedented” return on investment, he says.

Executives often say they use data to inform critical business decisions, but when push comes to shove, they fall back to trusting their gut instincts. But Alteryx CEO Dean Stoecker has a hunch that approach will wane in 2017.

“Chief executives will finally begin to trust data, not just their gut,” Stoecker says. “A 2016 study by KPMG revealed that one-third of CEOs have a high level of trust in the accuracy of their organization’s data and analytics, but 29% still have limited trust or active distrust. Data has traditionally been seen as a black box, but many organizations now have a “curator” of data—in some cases chief data officers or chief analytics officers—helping create a data-driven culture across the enterprise. Better workflow visualization has increased understanding of data at the highest level and more executives will continue to embrace it. To become truly insights-driven, we’ll start seeing more and more firms assigning data responsibilities to CIOs, CMOs and even CEOs, to drive swift business action based on data-driven insights.”

The unrelenting growth of data will force tech decision makers to take new approaches to data storage. According to DDN, 2017 will be a good year for object-based storage deployments.

The trust gap is still a big issue for big data in 2017 (eelnosiva/Shutterstock)

“In order to remain competitive, organizations will take a new look at their storage strategies and finally adopt more cost-effective storage platforms that include object storage that supports their peta-scale and soon exa-scale environments. There will finally be a realization of object storage as a stable, proven solution that is helping maximize storage density, increase utilization, reduce scaling complexity, improve performance and optimize TCO,” the company says. “We have been talking for years now about ‘the year’ that object storage will finally catch on. But as massive data growth continues to plague storage administrators, and with advances in integration and user acceptance, 2017 is finally ‘the year.'”

Cybersecurity breaches made big headlines in 2016, from Yahoo‘s two massive data breaches impacting 1.5 billion users to the hacking of the Democratic Party servers and subsequent disclosure of confidential information in an attempt to impact the Presidential Election. Thanks to these huge and public failures, security experts predict that organizations around the world will finally realize the errors of their ways and will invest billions to solidify their digital presences, helping the white-hats to gain the upper hand over black-hat types and making 2017 a breakthrough year for progress in cybersecurity.

Oh, wait. Scratch that. Actually, the security story gets worse—much, much worse. Moshe Ben Simon, Co-Founder and Vice President, TrapX Security, gave Datanami these six security-related predictions that may have you yearning for the old days of when couriers roamed the streets. First, ransomware will grow to unprecedented levels. Next, the financial sector will lead the world in security losses. Ben Simon says we’ll see a marked increase in cyberattacks by nation states (Fancy Bear, anyone?), and that healthcare exploits will set new records. Attacks on IoT devices will surge, and last but not least, a successful cyberattack will take down part of the power grid in a major western country, he predicts. Fun stuff.

One of the factors driving the adoption of machine learning and artificial intelligence technologies is the desire of consumers to get the “celebrity experience,” as Forrester analyst Mike Gualtieri eloquently puts it. With billions of consumers out there, that’s a healthy potential market. But there’s another set of folks that are set to benefit from advances in AI and ML: Software developers. According to GitLab‘s VP of Product Job van der Voort, AI and ML technology will be a force multiplier for devs.

Cybersecurity figures to play a bigger role across many aspects of society in 2017 (Creativa Images/Shutterstock.com)

“Code will continue to get automated, resulting in fewer mistakes overall,” van der Voort says. “Devs will increasingly see aspects of their role automated, but in positive ways, such as code testing, gathering and formatting data, reporting and notifications. Coding via automation through machine learning will be more prevalent than years past, now made possible due to new hardware availability and techniques (GPUs, parallel computing). Despite the hype around AI, you won’t see AI replacing devs for at least 10 years, as AI that thinks for itself does not exist in this capacity and writing code is very difficult.”

We haven’t yet seen major advances in healthcare due to the application of big data analytics. But the huge amount of medical data we’re collecting increases the odds that 2017 may be a breakout year for healthcare analytics, predicts Fuzzy Logix COO Michael Upchurch.

“Sources of huge healthcare data sets are becoming more abundant, ranging from macro-level sources like surveys by the World Health Organization, to micro-level sources like next-generation genomics technologies,” Upchurch says. “Healthcare professionals are leveraging these data to improve the quality and speed of their services. Even traditional technology companies are venturing into this field. For example, Google is ploughing money into its healthcare initiatives like Calico, its ‘life-expansion’ project, and Verily, which is aimed at disease prevention. We expect the demand for innovative technical solutions in all industries, particularly healthcare, to explode in popularity” in 2017.

Where will your data reside in 2017? Will it matter? The physical location of data is becoming increasingly irrelevant as technology evolves to allow organizations to work with data wherever it resides. However, this explosion of data will bring its own set of challenges, says Steve Wilkes, co-founder and CTO at Striim.

“The lines that demarcate where enterprise data resides and is processed will blur rapidly during 2017 as more and more organizations extend their data stores to the Cloud and embrace IoT,” Wilkes predicts. “This will enable new opportunities and bear fruit in the form of scalability, agility and cost savings, but will be fraught with challenges concerning security, integration and analytics.”

Related Items:

Big Data Speaks: 10 Industry Predictions for 2017

2016: A Big Data Year in Review