How have diversity, salaries, and the tools and algorithms used changed in the field of data science from 2017 to 2020?
Since 2017, the website Kaggle.com has performed an annual survey of data scientists and others interested in the field of data science and machine learning. The survey questions range from demographic questions, such as gender and level of higher education, to questions about programming languages, tools, and machine learning algorithms used. A full description, as well as the results, of each year’s survey can be found on Kaggle’s website for 2017, 2018, 2019, and 2020.
Since it is the end of the year, and there are now four years of survey results, I thought it would be interesting to look at trends in the field over the past few years. New algorithms and techniques continue to be developed, so it will be interesting to see if newer techniques are replacing older ones, or simply being used alongside existing ones. …
Explainability and ethics in machine learning and AI should be introduced to aspiring data scientists from the beginning
Following the exploding popularity in machine learning, deep learning, and AI in the past decade, there have been increasing reports of occasions where AI solutions are not providing the expected results or have unintended consequences.
There have been numerous reports of bias in AI algorithms that adversely affect certain groups of people in serious ways.
There are also regulations now, such as GDPR, that call for the right of customers to understand why a company’s model gave a certain outcome for them.
There is a view among some practitioners that only “good people” should be working on these algorithms, to avoid problems of bias and other ethical issues. The problem with this argument is that even if we have only “good people” working on these algorithms (which, in many cases, we do), they still need the tools to look for these biases and understand where issues can arise. …
