Using Dark Matter Physics to Gain Mobile Intelligence

Before joining the data science team at Apteligent, I was a PhD physics student working towards searching and understanding dark matter. Dark matter is a fundamental new particle whose existence has been inferred by the physics community solely on its gravitational effects since the 1930s*. Unfortunately, any other information other than its probable existence (that’s right, we are not even sure if the particle actually exists) has been hard to tease out. We don’t even know its mass!!

Currently there are numerous experiments aiming to find dark matter. Due to the breathtaking sensitivity of the experiments and the numerous different dark matter models that need to be tested, scientists use advanced statistical techniques and machine learning to mine the vast amounts of data these experiments produce.

Note: Did you know that the world wide web was invented by a scientist, Tim Berners-Lee, while working on particle physics at CERN in Switzerland as a solution to sharing a lot of data quickly to his collaborators? Tim Berners-Lee made his invention open source so that we could all enjoy the wonders of the internet.

Apteligent retains its position as the leader in mobile app intelligence by utilizing many of the same big data analytic techniques that top international physics collaborations currently use in the search for dark matter.

At Apteligent, the data science team has built numerous predictive models using machine learning to give insight and intelligence to our mobile customers. As an example, we recently built a model that predicted the adoption of iOS 10 based on historical data. The plot below shows the forecast of iOS 10 adoption rate. Note that the iOS 10 adoption prediction resembles more the iOS 9 adoption after its release; as compared to iOS 8. This model accurately predicted that the iOS 10 adoption would cross the 50% threshold approximately 20 days from launch.

The particular parameters that comprise the model were found by Maximum Likelihood Estimation (MLE). MLE works by finding the specific parameters of the model with the largest probability to be ‘correct’ given the data. Usually an algorithm like gradient ascent is used to maximize the Likelihood function, and find the ‘global’ maximum which pertains to the best predictive model. MLE is a key concept in statistics and machine learning; it is widely used in multiple fields of research and investigation. In particular, it is also used by physicists to find the best dark matter model to explain the data; or conversely, to rule out the model completely**. The beauty of MLE is its flexibility to tackle many different problems.

Building predictive models with machine learning on big data is one of the biggest challenges of our generation, which pertains to many fields. Machine learning is currently used from creating the recommended movie list on your Netflix account and building autonomous cars, to researching cancer and predicting poverty in developing countries. The data science team at Apteligent harnesses the power of machine learning to mine approximately 100TB of data every month in search for new signals and indicators that help predict and give insight into mobile apps’ user-centric performance.

* Fritz Zwicky was the first astronomer to propose the idea of dunkle Materie ‘dark matter’ in 1933.

** A good book, written by my advisor, titled The Cosmic Cocktail: Three Parts Dark Matter neatly summarizes the journey the physics community has undertaken in its search for dark matter and its current challenges.

Originally published at www.apteligent.com on October 19, 2016.