‘Data Workers’ Failing to Cope

via Shutterstock

More evidence is emerging that “data workers” in general and data scientists in particular are bogged down by the sheer breadth of their company’s data.

Meanwhile, the skills gap between data experts and line-of-business workers continues to grow.

A vendor study attempted to gauge how much time is wasted during an average work-week fiddling with data pouring in from across multiple business processed and workflows. The IDC survey commissioned by data analytics platform vendor Alteryx (NYSE: AYX) found that data workers waste 44 percent of their time per week searching for and organizing data.

Adding to the frustration is data and analytics tool diversity. Alteryx has been the forefront of efforts to consolidate analytical tools ranging from data preparation to visualization. The IDC survey found that data handlers are dealing with an average of six data sources, up to 40 million rows of data and seven separate outputs. Outputs ranged from trend analyses and “insight sharing” to business projections and data applications.

(Data sources include, in descending order: spreadsheets; cloud, desktop and analytics databases; software-as-a-service applications; in-memory; relational database management systems; mainframes, “flat files”; NoSQL databases; and, bringing up the rear, Hadoop.)

Furthermore, data workers cling to their spreadsheets, with 88 percent acknowledging their continuing and widespread use in data projects. “Spreadsheet functions are often used as a proxy for data preparation, analytics and data application development tools but are error-prone and expose the organization to compliance and trust issues,” the survey concludes.

Alteryx and a growing number of data tool vendors argue that the more than 30 percent of the workday devoted to data preparation can be reliably automated.

“Consolidating platforms and looking for tools that address the needs of any data worker, whether a trained data scientist or an analyst in the line of business, can help reduce the friction that many organizations experience on their path to becoming data-driven,” said Stewart Bond, IDC’s director of data integration.

For its part, Alteryx cited the survey results as more evidence that companies need to consolidate disorganized data efforts onto self-service platforms such as theirs. The result, the company argues, would be greater automation of menial data preparation tasks and more confidence in the outcomes generated by focused data analytics.

A growing skill gap also is contributing to the data chaos. Analytical and statistical skills were most often cited by survey respondents as being in shortest supply as the talent gulf between data scientists and data workers widens. As a result, the survey notes, more companies are hiring chief data or analytics officers to shake up the ranks while attempting to streamline analytics processes.

“Collecting data alone won’t digitally transform a business and the answer is not as easy as hiring a leader, a few data scientists or over-investing in disparate technologies,” noted Alan Jacobson, chief data and analytics officer at Alteryx.

“The key is to empower all users, many of whom are currently stuck in spreadsheets, to analyze data effectively to drive real, business-changing results,” Jacobson added.

Recent items:

Alteryx Repositions for Analytic Tool Consolidation

Alteryx Goes Public as Data Prep Market Heats Up