Danger and Difficulty Temper Data’s Huge Potential

(Maksim-Kabakou/Shutterstock)

It has been called the new oil, the new currency, the new religion. It is data, of course, and it’s having a monumental impact on how we build business systems in the 21st century. However, for all the potential benefits that data bring can bring to organizations, many are discovering that data is also full of hidden pitfalls and risks.

Nobody should be surprised that we’re struggling with big data. While it’s true that we are in the middle of a great data boom that’s seeing data volumes essentially double every 18 months and the creation of technologies like deep learning that have huge potential, most organizations struggle mightily just to deal with the data that was already created, let alone start building the intelligent systems of tomorrow that will let our organizations take advantage of the even bigger and better data promised to come.

This exposes an interesting dilemma about the dual state of data today. On the one hand, data scientists are understandably excited about the potential to build AI-powered system that will unlock data’s potential and improve the quality of life for billions of people. But on the other hand, if we’re barely coping with the day-to-day drudgery of managing big data sets today, how can we expect to make giant leaps forward?

There is, of course, data to back up the seeming irreconcilable state of data today. Organizations are all over the map when it comes to the maturity of their data and AI projects. Some have already built solid data foundations for future AI innovation, while others are still searching for a good place to start.

Can’t Live With It…

Data security and governance present major headwinds to data productivity, particularly with Europe’s General Data Protection Regulation (GDPR) going into effect next week. Organizations that have instituted the proper controls for GDPR will be rewarded with the legal blessing to continue leveraging data analytics, while those that haven’t adequately prepared for GDPR must weigh whether the legal exposure of GDPR is too great to continue with analytics and AI.

(Robyn-Mackenzi/Shutterstock)

The European Commission isn’t the only group that will punish companies that mismanage their data. According to a Veritas-commissioned survey 12,000 people, nearly two-thirds say they would stop buying from a business that fails to protect their data, while almost half say they would abandon their loyalty to a particular brand and consider turning to a competitor if they mismanaged their data.

What’s more, 81% say they would tell their friends and family to boycott the organization, while nearly 74% say they might report the business to regulators. And 65% say they would post negative comments about the business online. Considering the swiftness with which bad news travels across social media, companies should have ample incentive to make data security a priority.

“As consumers demand more transparency and accountability from businesses, the ‘new norm’ will see consumers rewarding those organizations that have good data hygiene practices in place while punishing those that don’t,” says Tamzin Evershed, a senior director with Veritas and its global privacy lead. “Businesses must be seen as trusted custodians of data if they want to reap the rewards associated with building consumer confidence.”

But even beyond security, big data poses a risk to profitability for those who wade into the analytics waters unprepared. A new survey of nearly 300 data professionals conducted by Trifacta concludes that companies are wasting billions of dollars per year on manual data preparation processes.

The survey found that 60% of IT folks are spending 50% or more of their workday cleaning or preparing data or doing some other data quality task for data analysts. A majority also found that data analysts are dependent on IT for data prep and to get access to data.

The impact of all this adds up to $450 billion being spent annually on tasks surrounding data preparation, data cleansing, and data access, Trifacta says. This inefficiency drains resources and prevents organization from value from analytics, says Trifacta, which develops machine learning-based tools to automate data preparation.

… Can’t Survive Without It

And yet, despite all the exposures, risks, and inefficiencies inherent in data analytics projects, the upside of getting data right is so potentially great that it drives us forward. And of course, there is yet more data to back that up.

Gartner said last month that AI will generate nearly $4 trillion in business value by 2022. Much of this growth stems from the expectation that deep neural networks will significantly improve the ability of robots and computer programs to make decisions on their own and interact with we humans people in a pleasing and profitable manner.

(Robyn-Mackenzie/Shutterstock)

The smart folks at McKinsey & Company say AI has the potential to increase top-line revenue by up to 12% over the next decade, which is a huge increase. But the bottom-line profit line should go up even more according to Corinium Digital, which recently released a report titled “The Innovation Game” that found that AI could post profitability by an average of 38% by 2035.

Advanced analytics and AI have the potential to help companies create real value in the coming years, including accelerated drug discovery in pharmaceuticals, tighter replenishment schedules in wholesale distribution, better risk management in life insurance, to name a few. The potential advantages AI are so great that not only will it reward those who successes at digital transformation, but it will likely push those who get it wrong into oblivion.

Data lies at the heart of this transformation and provides the fuel to generate meaningful insights that could be worth $14 trillion by 2035, Corinium writes in its recent report, which was commissioned by Microsoft, Paxata, and Accenture.

“We have reached the tipping point where all businesses recognize they cannot compete in a digital age using analog-era legacy solutions and architectures,” the company writes. “The winners in the next phase of business will be those enterprises that obtain a clear handle on the foundations of modern data management, specifically the nexus of data quality, cloud, and AI.”

Organizations that succeed with data and AI projects would do well to acknowledge this dual nature of data – dangerous and volatile in one moment, and indispensable and hugely profitable the next. Like ideas and nuclear material, those who can successfully harness the power of data while managing the very real risks will be the most successful.

Related Items:

The 10 Step Guide to Mastering Machine Learning

Jeff Dean Thinks AI Can Solve Grand Challenges–Here’s How

What’s Challenging in Big Data Now: Integration and Privacy