Big Data Job Market Unexpectedly Cooled in 2014

It was time for a reality check. After sizzling for two to three years, the big data job market hit an unexpected dip in the second half of 2014, according to a report from Foote Partners, which found the market values for 58 big-data related skills declined by an average of about 5 percent.

The big data decline began in the second quarter of 2014, when organizations started paying less for certain skill sets, according to Foote, which tracks the marketability of hundreds of IT skills in its “IT Skills Demand and Pay Trends Report.”

“The problem that developed in 2014 is too many employers have not been satisfied with the return on their sizable investments in big data initiatives,” says David Foote, chief analyst and co-founder of Foote Partners.

According to Foote’s survey of more than 200,000 IT workers in the U.S. and Canada, pay premiums for 31 noncertified big data skills (such as Hadoop, MapReduce, Hbase, Hive, NoSQL, data mining, and base SAS) fell an average of 3 percent in over the second half of the year, but still ended up slightly for the year, thanks to healthy gains from January to June. Pay for 37 big data certifications (such as those from Cloudera, MongoDB, Teradata, and others) grew more than 10 percent for the year, but lost some value as the year went on.

The drop came as a surprise, Foote says, but it should be a reality check about what it takes to succeed with big data technologies. It takes more than possessing data, analytic tools, and skilled people to succeed–you also need the right organizations structures in place to benefit from big data.

“Holding back these [big data] returns have been mostly organizational and cultural barriers related to transparency, data governance, and sharing of data enterprise wide,” Foote notes. “The fact is that there are barriers that don’t normally exist in companies with less hierarchical management and decision-making structures. You can’t simply hire Hadoop experts from Google and Amazon and expect to replicate the big data capabilities found at those companies.”

You could call it a market correction. The rapid onset of demand for people with big data technologies easily outstripped supply over the past several years, which caused their salaries to rise. People who have Hadoop, MapReduce, Hive, and HBase skills found themselves able to command six-figure salaries. Meanwhile, full-fledged data scientists–which the Harvard Business Review in 2012 famously labeled “the sexiest job of the 21st century“–could write their own tickets, and were often the subject of bidding wars.

The big data job market may have slowed a bit, but the overall trajectory for big data skills remains positive. Companies are taking a more measured, long-term approach to their big data investments, Foote says, including doing more with the data and the personnel they already have.

“We expect big data skills market values to recover slowly over the next 12 to 24 months,” he says. “Big data capabilities are just too critical for staying competitive.”

Related Items:

9 Must-Have Skills to Land Top Big Data Jobs in 2015

Rise of the Big Data Engineer

Plugging the Big Data Skills Gap