The Evolution of the Data Scientist

The role of the data scientist is currently one of the most in-demand jobs in the tech industry. As more businesses turn to big data analytics for insights into their customers, trends in their industries and to gain a competitive edge, this role is constantly evolving and moving from obscure to mainstream.

Coined by Jeff Hammerbacher and DJ Patil in Silicon Valley in 2008, the data scientist is facing new challenges as the information this individual works with is growing both in volume and variety, which provides additional layers of complexity. It demands an individual who is able to deploy different techniques to mine through data in an unconventional way, a ferociously smart individual who can look at data sets with an open mind and creative zest.

When this profession first emerged, there were no specialized degrees or training programs available. With the growing increase in demand, programs across the country are starting to develop to fill this need, both in the undergraduate and master’s levels. The evolution of these programs, with computer or library sciences giving way to specifically data focused courses of study, is very telling of just how important this role is becoming.

Two Halves of a Whole

There are two distinct paths one can take when entering the world of big data. Both development and analysis play equally important roles and cannot function without the other. To put it into perspective, let us consider how a race car team operates, with multiple parties performing various but equally important tasks.

The developer role focuses on creating the infrastructure, platforms, and tools that enable the gathering, storage, and eventual analysis of data, such as Hadoop. Consider a developer to be an engineer who designs a top-of-the-line car engine. This individual focuses on speed, precision and accuracy–logistical details that allow for the next person to do their job.

The analytic side of things focuses on the slicing and dicing of data and the information that is derived from it. This requires strong critical thinking and problem solving and can be equated to the role of a race car driver. This individual has to identify the problem analytics has to solve, what data should be looked at and how the results should be applied across the business. He or she has to deploy the platform and ensure it is being used to its highest capability.

For someone looking to enter the field, they must determine what kind of role they want to play. Do they want to drive the Ferrari or build the engine that powers the machine? Without a good engine, the driver cannot win the race. On the other hand, a top-of-the-line machine will collect dust unless a capable individual can run it.

Where Do We Go from Here?

The role of the data scientist is currently at a crossroads in many ways. Big data analytics continues to grow at a tremendous rate and the implications of that rapid growth are constantly changing. This causes multiple parties across the enterprise to develop techniques to cope, which puts the role of a data scientist in an interesting and somewhat contradictory position.

We are already seeing a rise in educational resources dedicated specifically to the data scientist. Institutions will continue to respond to this demand by churning out dedicated professionals with specified training, equipped to address the analytical needs of marketplace. Some might also choose to continue pursuing education to keep their skills sharp and to keep up with technological advances needed to stay relevant in a rapidly evolving industry.

In the meantime, the enterprise will also react to this growing demand, identifying an opportunity to capitalize on a predominant need in the marketplace and look to provide a solution. As we move forward, we will see the creation of tool sets designed to empower the average employee to work with data in order to derive the necessary information.

We’ve seen this evolution happen before. When the automobile was first invented, the role of the driver was highly valued as maneuvering and controlling the vehicle was rather difficult. In order for consumers to take advantage of this new technology, this specialized individual was necessary. As Ford continued to produce cars, it realized it could sell product to more consumers if they found the product easier to use. By eliminating the middle man and making driving accessible, Ford was able to market its product to a large audience and empower everyone to become a driver.

To bring this analogy to the present, let’s take a quick look at website development. When the World Wide Web exploded onto the scene, Web developers were the hottest commodity. But over the years, coding was simplified and anyone willing to learn could spend some time studying it. Now, creating your own website or blog is relatively simple.

Looking Forward

Over the next four or five years, demand for data science skills will increase and cause data analytic technologies to become mainstream. Companies will need to work with analytics in order to stay relevant and to compete in their respective industries. What was once seen as complicated and out of reach will now be impossibly practical for companies to execute. Big data will no longer be exclusive.

While the role of the data scientist will not disappear, it will definitely evolve. New tool sets will give the “Average Joe” a chance to dabble in data, but he will not be able to do it all. Fueled by vast amounts of data we will be collected from the Internet of Things (IoT) and other machine to machine sensors, data scientists will require an even more specialized set of skills to continue to innovate and make use of all the incoming information. This role will become even more intertwined with the IoT, and the creativity and open-mindedness valued in today’s data scientists will prove to be even more crucial as we continue to innovate.

About the Author: Don DeLoach, CEO and President of Infobright, has over 25 years of software industry experience. Prior to Infobright Don was CEO of Aleri, the complex event processing company, which was acquired by Sybase in February 2010. Prior to Aleri, Don served as President and CEO of YOUcentric, a CRM software company, where he led the growth of the company’s revenue from $2.8M to $25M in three years, before being acquired by JD Edwards. He has also served as a Director at Broadbeam Corporation and Apropos Inc. Don is active in community service, and is a Director on the board of the Illinois Technology Association and the Juvenile Protective Association in Chicago. Don has a Bachelor of Industrial and Systems Engineering degree from the Georgia Institute of Technology.

Related Items:

Masters of Big Data: New Graduate Programs Debut at U.S. Universities

Six Big Name Schools with Big Data Programs

Time to Query Your Data – Do You Know Where Your Data Scientist Is