What is the future of data analysis?
1. Emergence of systems that can better reconcile the ability to scale with the ability to handle the analysis of linked data
2. Analytics technologies will be able to abstract out a lot of the effort that needs to be put into choosing what best algorithms to use for modelling and visualizing the data, turning them into computational optimisations, allowing analysts to focus more on the higher level questions , like what they want to ask from the data, how they want to merge and group the data, and what do they want to remove from the data, which may give more value for analysts with domain knowledge than those with statistical knowledge.
3. The improvement in speed, scale and reliability of analysis technologies, coupled with the increase in the degree of connectivity of the world, where at some point any atomically identifiable entity in the world will have a representative equivalence on the internet, will allow narrowing the gap between when the data is collected, analysed and acted on in the offline world not just online. Ideally creating a virtuous circle where insights extracted from the data can be immediately fedback for making decisions, that themselves change the nature of that data on the fly, and so on(that may already be the case in financial markets but other areas are yet to catch on). If that were the case, the ability to make decisions will be a critical skill for data scientists, not just the ability to handle, analyse,visualize and deliver data.Read more