Navigating the Intersection of Finance and AI
Jessie: Welcome to OxAI podcast, I am your host Jessie from the Oxford artificial intelligence society. Today we had the pleasure to have Chak Wong, Global head of time series reinforcement learning at J.P. Morgan’s Machine Learning Center of Excellence and Professor at Hong Kong University of Science and Technology. Chak is also an Oxford alumni graduated with a DPhil in Economics. Welcome to join us today, Chak!
Before we start, congratulations on winning the Global Technology Innovation Award in 2023! This is a great accomplishment!
Chak: It’s the joint effort of the team.
Jessie: Do you mind sharing about your work and research?
Chak: Decision making is a completely different task from forecasting. Forecasting is a more common task, which is what we are used to in traditional statistics methods. However, decision making is a different story. My daughter is applying for university this year. She’s applied to Oxford, and I do hope she gets in. However, although some, for example, forecasts say that her income can be 20% higher after graduation, some may suggest otherwise. A survey has shown that the top-tier university might not be the optimal choice. There might be other options she can explore that ends up with higher income. Good people gets accepted to Oxford and good people gets paid higher. It may have nothing to do with entering Oxford. So forecasting is for a single point in the future, but decision-making takes into consideration any other options that may be relevant.
In addition, causal relation is crucial in the decision-making process. Here’s an example, data has shown that people vaccinated against Covid have a higher fatality rate than people without vaccination. But is this the case? Probably not. This is because people with underlying health conditions are prioritised for Covid vaccines, so there is selection bias in the sample. There are countless examples of this. We use causal inference to identify these relations.
Our team explores machine learning methods to support decision making. We research causal inference, decision focused learning, time series analysis and other related topics for this problem. At the same time, reinforcement learning becomes a natural optimisation tool after we build an environment that respects the causality relationship.
Jessie: This naturally leads to my question. You have a solid background in quantitative finance and econometrics, and now you are leading the frontier machine learning research into this space. What advantage, would you say, can AI / large ML models bring compared to traditional methods? And what is the difference between the ML methods and the traditional econometric methods?
Chak: Under traditional econometric approaches, we would fit a model for prediction or optimisation etc given data available. However, the data we train on might not be relevant to our purpose. Taking an optimisation example, if we want to optimise the route from Oxford to London, training on data with routes from Paris to Moscow or routes from Hong Kong to Sydney would not help. We would like to have a specific model for each individual problem.
This is difficult to be achieved using traditional econometric models but can be tackled with machine learning methods. One approach in ML literature is to consider a differentiable optimisation layer in the neural network. The constraint optimization problems within the network can be parameterized in a way that allows for gradients to be computed with respect to the parameters. This makes it possible to obtain problem specific solutions even if the NN is trained on a large dataset. One of the pioneering papers in this area is ‘OptNet: Differentiable Optimization as a Layer in Neural Networks’.
Jessie: As managing director at JP Morgan MLCoE and Professor at Hong Kong University of Science and Technology, you have seen both the research development and the industry use cases. What would you identify as the biggest gap in literature and industry application? And what do you think is causing it?
Chak: In academia, the research questions are usually well defined. This is rarely the case in industry. The question you are facing might just be ‘how to increase profitability?’. Or someone might just come to you and ask, ‘how to be happy?’. This is a philosophical question. To answer this from a statistical approach, one can conduct a survey on people’s level of happiness and fit a model to discover the driving factor. The ability to break down a problem and translate it into specific research questions is essential.
Moreover, in academia the methods are developed, most of the time, under ideal conditions or under certain specific assumptions. This is hardly the case in actual implementation. When developing a method, the model robustness is a key focus. It may be more than half of the research. For a newly developed model, its robustness needs to be understood mathematically, and tested via both simulation and experiment on real data, and afterwards on small scale experiments.
Jessie: What is the challenge in applying AI in finance, especially quant finance?
Chak: This is an industry that requires an interdisciplinary background, it requires knowledge in finance, economics, statistics, mathematics and computer science etc. This is not simply having multiple people in the room coming from different disciplines. One has to at least understand each field to be able to come up with novel research ideas.
Jessie: What would be your one piece of advice for Oxford students who are interested in starting their career in finance and AI?
Chak: I would recommend focusing on the fundamentals. I have interviewed a lot of students. What I observed is that there’s a lack of knowledge on the basic concepts. For example, if you train a model with MSE loss, what does this imply on the loss function? And if you change the loss function, how would you expect the model to behave? Another example is, if we would like to do inference on a time series, what are the assumptions to check before fitting any time series model? Most people would fit a LSTM directly. A model without its assumptions fulfilled cannot produce sensible output. Always analyse your data, check the model assumptions before fitting any model!
