Data Scientists–Who Needs Them Anyway?

It’s great being data scientist. You get a ridiculously large salary and, because the Harvard Business Review called it “sexiest job of the 21st century,” automatic success on the dating scene. The problem is, like the Eastern Lowland Gorilla and the Black Spider Monkey, data scientists are an endangered species. Good luck finding one in the wild.

What a data scientist might look like.

The shortage of people with data science skills–math, statistics, programming, critical thinking–has been well-documented. In 2011, the McKinsey Global Institute published a study that found that, by 2018, there could be a shortage of up to 190,000 data scientists in the United States, representing a 50 percent to 60 percent gap between supply and demand. The shortage is so pressing that Greylock Partners, the early-stage venture capital firm funding big data startups, created its own recruiting team to match data scientists with companies in its portfolio.

Luckily, the nation’s universities have heeded the call for more data scientists by stepping up production. In the past few years, more than two dozen universities–big schools like Columbia, Stanford, Northwestern, Syracuse, UC Irvine, and Indiana University–have launched graduate programs aimed at training people in advanced analytics and other data sciences. That will help ease demand for top data science talent from established programs, such as the one at North Carolina State University. However, even North Carolina State only produces 80 data scientists out of its program, which accounts for much of the estimated production of 300 freshly minted data scientists each year out of our nation’s higher education system.

Without a suitable population of data scientists to fill the jobs that are available–and no likely easing of the supply crunch in the foreseeable future–companies have been forced to get creative, and to take it upon themselves to address their data science needs in other ways. In some cases, they may try to outsource that function (difficult when strategic decisions are being made) or turn to software to make up the difference.

But there’s a third way: lowering your standards. If you can’t hire a data scientist because there are none who will work for less than $350k a year, a Cadillac health plan, and a really, really good parking spot, then do you have any choice but to go down stack and re-evaluate your needs? The answer, according to the folks at big data software company Alteryx, is: absolutely not.

Companies are beginning to find that they don’t need data scientists after all, says Rick Schultz, the senior vice president of marketing at Alteryx. In many cases, companies are taking second looks at data analysts–who live one or two rungs down the data science ladder–and are saying “That’s good enough for me” and even “Great, I’ll take two.”

“It may seem controversial because of the amount of attention that’s been paid to data scientists,” Schultz said in a panel discussion that took place on the Internet earlier this week. “On the other hand, this has already started happening over the last couple of years. What’s changing as we enter 2014 is the pace of the shift is increasing.”

Schultz doesn’t dispute that, in a perfect world, having a highly skilled data scientist on staff would be ideal. “But there’s just not that many of them,” he says, “and they tend to be much more expensive resources as well. When you think about the requirements for who you want performing that data analysis that will inform your most strategic business decision, well guess what? That person is already on your team. It’s your data analyst.”

The great thing about data analysts is there’s a lot of them. Like the burgeoning black bear and white tailed deer populations, there are plenty of decent data analysts prowling American’s corporate woodlands, ready to apply their business-focused acumen in a data-intensive setting. So what if they aren’t classically trained Bayesian statisticians? When life gives you lemons, you gladly take that lemon-flavored data analyst.

According to Schultz, the active population of data analysts in the US and Europe is about 40 to 50 times bigger than the population of data scientists. When you look at the job opportunities available to data analysts, you find that there are about 25 times more open positions for data analysts than for data scientists, he says.

“That’s not even accounting for the fact that a lot these analysts don’t even have ‘analyst’ in their title,” Schultz added. “Many go by business analyst or data analyst. But others go by market planner or marketing operations.” (Or senior vice president of marketing, we suppose.)

The numbers still favor the rare data scientists, who are still more heavily hunted relative to their scarcity in the wild, and who still command giant salaries and great parking spots. If you’re on the demand side of the equation (i.e. you need someone with data analytic skills), you’re still using the freshest bait to lure in the big data game, or perhaps filling your limit on data analysts while they’re still in season.

If you’re on the supply side of the equation (i.e. you have data science or analytic skills), now is a good time to think about enrolling in one of those data science programs and be on the cutting edge of the next generation of big data analytics. Don’t dally: a big salary and the adoration befit of a true nerd awaits you. If you’re good with numbers but don’t have the raw brainpower or academic credentials of a true data scientist, early 2014 might be a good idea to test the employment waters, and possibly look at adding “analyst” to your title.

Related Items:

Salaries Account for Nearly Half of Big Data Spending

Raising a Pack of Data Scientists

Visualizing the Big Data Job Market