Software is eating the world, including data science.

As the prolific investor and entrepreneur Marc Andreessen put it, “Software is eating the world.” While some may find this a cautionary tale of a dystopian future eating valuable jobs, the opposite is actually the case. Nearly every industry is positively influenced by software. Innovative companies that create great software disrupt antiquated and stale industries, and if anything, increase demand for these improvements. These companies can sometimes create entirely new industries and jobs that would have never existed just a few years ago. In essence, there is a symbiotic relationship between software development and job growth.

Data science has evolved from being primarily an IT initiative into business applications where it actually adds value. Innovative software companies have disrupted the data science industry to give rise to more jobs and more competition in the data science, machine learning and even data visualization space. The barrier to entry for companies to get into the analytics game has been greatly reduced in the past decade as a result of software advances. With technical boot camps, MOOCs and a good statistics background, anyone can become a data scientist. One thing to emphasize is that data science is the furthest thing from a new concept; it’s fundamentally statistics. What has changed is that software has enabled engineers and business owners to leverage this knowledge into powerful applications. The data science field is growing rapidly with vast improvements in technology and software, and many of the tools are free. Tools like R and Python are popular within the data science community.

McKinsey Research estimates that there will be a severe job shortage of 190,000 by 2018 to meet this demand. Most data scientists can command an average salary of $122,000, according to KDnuggets. With technology always evolving, we must rely on software development to help grow the data science field to automate many tasks in a data scientist’s workflow.

To a certain extent, job demand will follow the current hype until software automates it. You don’t hear anyone claim there is going to be a job shortage of 200,000 steam engine mechanics to help aid the modern Industrial Revolution, do you? Of course not. But this example illustrates that job demand is largely driven by hype and demand at the time. Even several years of software and hardware innovations can make current jobs obsolete.

The “severe shortage” that is predicted in the near future will actually not be the case because of three reasons:

1) Software in the data science industry will gradually evolve to automate redundant or archaic processes now, thus relieving pressure to hire more personnel with data science talent in the future.

2) Training will lower the cost for entry for future potential candidates. Bootcamps, MOOCs and even the modern collegiate programs are embracing this skill set.

3) It’s popular to be a data scientist right now, but something else will pique organizations’ interest such as Machine Learning, Computer Vision, Artificial Intelligence, Internet of Things and Augmented Reality. Even though the workflow of these positions can be nearly identical to data science, the hype will transform it into something else.

In the near future, data science will largely be considered a field rather than a singular position. Acquiring software will be extremely valuable in addition to hiring the right positions in a data science team. A data designer is a new job title that will enter the market in the coming years. They are tasked with communicating complex data workflows closer to a graphic designer, rather than a data scientist, since it will likely require minimal statistics knowledge. The majority of data scientist initiatives is just hiring someone that can communicate the value of advanced analytics. Data analyst, data engineer and data designer will all be positions that will gain traction and stem from the core data science field.

Here are several recommendations before embarking on a data science initiative:

1) Don’t reinvent the wheel. Many other organizations may have already created great software to solve complex data science problems. Not Invented Here (NIH) plagues modern businesses.

2) Subject matter expertise and communication are by far the most important factors in a successful data science hire. Hiring the right tech talent with a passion and knowledge in your industry will increase project success exponentially. Don’t hire a jack of all trades, as they will be a master of none.

3) Instead of hiring amateur data scientists, rent the software instead. Many disruptive companies are advancing data science that will enable business analysts to create self-service data science reports, a term called Citizen Data Science. Use your data scientist FTEs to solve complex problems and use software to automate redundant workflows.

These are exciting times for data nerds, but now the difficulty in this market will be for the right talent to stand out to employers. What will be difficult for an organization to determine is when to hire or buy software. As time goes on, more data science software vendors will evolve and automate the workflow of data scientists. Many companies such as Continuum Analytics (makers of Anaconda), Dato (makers of GraphLab) and Microsoft’s Azure Machine Learning are already creating great solutions to automate data science workflows. Software may be eating the world, but it’s creating greater job opportunities in data science, not job shortages.