Data can help us understand humans, the world and why things happen
Many services and processes nowadays are based on data analytics: finance markets, health services, marketing, risk management, predictive maintenance, and many others. Quite often, the use of data is focused on specific targets and goals — e.g., using specific data to calculate a score and propensity to take certain actions. And it makes sense in many cases. Yet, as we know, we can lose the big picture if we only focus on details. Could we use data better and more holistically to understand the world, human beings as a whole, or how different things impact each other?
Nowadays, we use data to analyze how marketing campaigns impact sales, how sales (both yours and your competitors’) impact stock price, how weather affects electricity consumption, how interest rates impact currencies and real estate value, and many other things. We also know that these are simplified models, and they don’t explain everything. Still, they are easy to implement and we believe they explain enough.
I wrote earlier about how we know some things have an impact and some don’t, but even when we know something has an impact, we cannot always utilize that information. It is clear that most models are simplified versions of reality. It is, of course, practical to have simplified models that are easier to understand and implement and explain many outcomes. But at the same time, it is crucial to understand that external factors can make our simplified models useless.
At a time when the capability to collect and analyze data is growing rapidly, would it be feasible, practical, and valuable to also have more complex models to understand larger things in the world? Let’s take a look at some examples.
Understanding weak signals
1. The financial market
This is one of those areas where companies put many resources into analyzing and predicting things better. A lot of tools and analyses still focus on data from other financial market instruments and economic metrics. However, there are already some hedge funds that try to analyze much more data, including some less conventional data.
For example, if people start to buy more canned food, the number of mental health cases grows or there are more accidents in a certain area. It is unclear if those factors impact the finance market, but data analytics can find which factors are relevant and amplify weak signals.
2. Governments and public safety
Governments must analyze security risks for public safety and security. Security services collect a lot of data. They especially try to focus on known threats, such as the behavior of hostile neighboring countries, the activity of terrorist cells, and the number of suspicious persons attempting to enter the country. Fire safety authorities want to analyze flammable materials in buildings and risks in high buildings.
Nowadays, much more data is available to at least detect weak signals regarding a threat level. For example, the demand for explosive ingredients, unusual usage patterns in mobile networks, changes in traffic patterns, or cyberattacks can indicate preparation for something else. Fire authorities could systematically collect and analyze data from all fire incidents, sales of risk items and changes in risky behavior. In this way, it would be possible to react to many risks much earlier.
3. Health and wearables data
Today, health apps mainly focus on daily changes of individual things like the correlation between our sleep and heart rate. However, by combining many data points and external data, we could also better detect long-term changes that can, for example, predict more serious illnesses or risks in our behavior and environment. These are things that daily changes fit into an average variance. But when we start noticing a constant shift in small things, it can indicate some bigger underlying factors. This could also enable better tools for preventive health care or identifying risky changes in external factors.
Those are just a few simplified examples. Naturally it’s hard to offer exact examples of possible data sources when the whole idea is to find a lot of data and then, based on the data analytics, see which data and factors have an impact and which are meaningless. Sometimes this kind of analytics can also find root causes of some phenomena, while the current models merely catch the more visible consequences of the underlying causes.
There are many counterarguments for this type of big data analytics. We can argue that it is a waste of resources, especially if we catch 80% of important things with much simpler and targeted models. We can also say that using that much data raises privacy issues and can even lead to a ‘Big Brother’ controlled society. And we can also argue that we will never be able to find all data that might be relevant in some situations.
Analyzing the ‘big picture’ to make better decisions
That is all true. But there are still many cases where more advanced analytics with richer data can offer real value for businesses, governments, society, and individuals. Weather and climate models are now one example, where supercomputers are used to model tons of data. It is very valuable work for weather forecasts and understanding climate change. In the same way, we could start using data to better understand many other complex phenomena in the world.
The reality is that lots of data is already available, even publicly. We must also divide applications into those that are more tailored for individuals vs those that analyze larger systems. For individuals, e.g. for personal health, the solutions should be user-centric with user-held data, with the capability to combine public data to personal data on the user’s applications, while ensuring the user’s data is not shared elsewhere.
For finance markets, public safety and security, many things can be done with more generic data, with no need to include individual user data. Often, a broader analysis of big datasets can decrease the need to analyze individual people. For example, if you can profile relevant risks in a society better, you have less need to track individual people, which is very different from models where governments track and score individual people.
The amount of data, data sources, and processing power are growing rapidly, which enables much bigger data analytics technically and economically. This can offer many ways to make our economy, services and daily life better and enable us to live healthier lives. This is similar to how a company must understand its market and have a strategy, and not only react to individual events.
It doesn’t have to mean a police state — the focus should be on understanding humans, societies and businesses to empower them to make better decisions. I’m sure these kinds of ‘bigger picture’ data solutions will be a growing business trend soon.
Connect with us and stay in touch
Prifina is building resources for developers to help create new apps that run on top of user-held data. No back-end is needed. Individual users can connect their data sources to their personal data cloud and get everyday value from their data.
- Follow us on Twitter, Medium, or LinkedIn.
- Listen to our podcast.
- Join our Facebook group Liberty. Equality. Data. where we share notes about Prifina’s progress.
- You can also explore our GitHub channel and join us at Slack.
