May 1, 2013
Budding Data Scientists Lack Access to Big Datasets
As noted in yesterday’s piece on re-evaluating IT hiring performance metrics, there exists a gap between big data worker demand and supply. Teradata explored the issue in its third annual State of Business Intelligence Survey.
The simple response is that, like the world’s data, the demand for data scientists and architects outpaces the ability of universities to produce them, or at least produce them effectively. One of the challenges lies in recruiting those who can educate potential data scientists, as those with the ability are likely tied up with data analysis duties. Indeed, the survey reflects this notion as it noted that “qualified or available faculty” was the 3rd largest challenge in teaching data science.
However, the primary reason why the gap exists can be well summed up by the opening sentence of the report: “Job prospects abound for college students seeking careers in business intelligence or analytics, but the greatest challenge to filling the talent pipeline lies in students having access to large data sets as part of the educational experience.”
Of course, universities employ data analysts who double as professors. For those, the top two challenges (access to large data sets [45 percent] and finding students with the pre-requisite skills [39 percent]) roll into each other to a certain extent.
There is a marked difference between the typical programming taught in computer science courses and that required in big data analysis. Computer science majors learn to deal with many of the languages of big data, such as Java, C, Python, and others, but the new big data techniques lie somewhat out of reach. Theoretical approaches to such education are helpful, but do not facilitate the trial and error processes that access to a large dataset would.
As a result, access to large datasets for educational purposes will surely rise over the next few years as it becomes essential in further growth in the big data space. “What faculty are looking for today is access to real, big-data sets. They want to show students the impact of the data explosion, demonstrate the linkage between data and business outcomes, and teach exactly how to achieve those outcomes,” said Barb Wixom, study author and Associate Professor of Commerce at University of Virginia’s McIntire School of Commerce.
On the bright side, the desire in the student population is palpable. According to the report, 40 percent of students surveyed identified “data-savvy careers such as business analyst, IT professional working with analytics, or a role in business that requires an understanding of data analytics” as a prospective career path. That includes 22 percent who want to use such skills in marketing, 20 percent in finance, and 16 percent who want to progress directly to data science.
Related Articles
Visualizing the Big Data Job Market
Leading Solution Providers
Tabor Network
