How we used elixir to boost our XIRR computations by 7x
By Shubham Gupta.
At the time of writing, Scripbox manages Mutual Fund assets well over Rs 700 crores. Portfolio metrics are crucial for our investors to judge and justify the decision of trusting Scripbox with their wealth. We use a modified form of Internal Rate of Return called XIRR to achieve this.
Jump to the technical write-up if accounting math makes you sleep.
Compounded Annual Growth Rate (CAGR)
Imagine that you have put some money into a bank fixed deposit. Typically, the principal earns interest over time in a compounded fashion and on the maturity date you get back the principal with the accumulated interest. The rate of interest is essentially CAGR.
CAGR makes sense when you are tracking the journey of a single investment. Typically you only need the Maturity value, Principal and time taken invested to calculate CAGR. When you have multiple cash inflows or outflows throughout the invested period, you can calculate the CAGR for each of them but not for the entire invested corpus. Internal Rate of Return is used in such cases.
Internal rate of return (IRR)
This is a metric used by companies internally to evaluate future projects. From a company’s perspective, instead of depositing money in a bank and getting annual returns, it can invest in itself (newer projects) and reap higher returns on invested capital. Hence the term internal rate of return.
Investments in a project would require an initial cash outlay and potentially investments later on to keep the project afloat until it reaches its objective. From a cashflow perspective, these are negative cashflows (the company is paying from its pocket). The returns the company receives from the project will become positive cashflows. IRR is great in accommodating cashflows. The catch is that every cashflow needs to be spaced a year apart. It is a good tool to estimate the viability of the project far out into the future, but not good enough if you want to tweak the time lever.
Enter XIRR. We don’t know what X stands for, but most likely it could just mean MS Excel as the name comes from the eponymous Excel function. When you have cashflows that are staggered across the year you need XIRR. The mechanics are similar to IRR.
You might be thinking at this point what has this got anything to do with portfolio reporting for mutual fund investments. Well, if you are a careful reader, you would have already made the connection between cashflows and investments. When you invest your bank account is debited (negative cashflow) and when you withdraw, your bank account is credited (positive cashflow). When you have a series of such cashflows, XIRR computation is possible as you have both the values and dates.
The formula for XIRR is as follows:
Don’t panic.
Our challenge here is to solve the equation for XIRR, i.e find the roots. What we have with us is a series of cashflows with dates and amounts that we need to plug into the equation.
There are multiple methods to find the roots:
Bisection Method
This method estimates the roots of a function by recursively reducing the size of the interval by half. An interval denotes two guess values for XIRR that yields function values on the either side of the x-axis. A detailed explanation is given here.
This method is almost a brute-force approach to obtain the solution. Although it’s very simple, it slows down quite drastically as the number of cashflows increase. This is to do with the increasing order of the polynomial.
This proved to be a showstopper for us as it was not a scalable solution.
Newton-Raphson Method
This method is used to find successively better approximations to the roots (or zeroes) of a real-valued function.
In other words, find the roots faster.
The Newton method uses a neat trick called the derivative of a function to obtain the roots faster.
Essentially, a derivate is the slope of the tangent to a line/curve at any given point. It is the best possible approximation of the value of the function at this point. We can use this method to find approximate value of the function at every possible point.
A derivative points in the direction of steepest ascent of a function. Hence, to find our roots for a function, we will go in the negative direction of the derivative. This direction should point to the root.
The Newton method starts with a function f with a derivative f’ and an initial guess value x_n then we get a closer approximation x_{n+1} as:
We iteratively apply the above formula until the difference between consecutive roots is less than a threshold value (called epsilon).
The Challenge
In our Rails application, we were using the Newton library to compute the XIRR for all our investments. This was a nightly job that was executed in sidekiq. Our implementation had the following drawbacks:
- We would calculate XIRR for one customer at a time. Hence, the amount of time it would take to compute XIRR for everyone was increasing rapidly as more users were joining the platform.
- Since this was computation intensive, we often faced server crashes that needed manual intervention to re-queue the jobs.
- Each investment was processed in a single thread. Due to this, XIRR for customers with large number of cashflows would take significantly longer.
The Rationale
Elixir was our natural choice to replace our existing ruby implementation as:
- It had inbuilt concurrency: we do not need a dedicated background processor for our asynchronous jobs.
- Elixir syntax is similar to ruby so we had a smaller learning curve to overcome.
- There were existing support for packages which compute XIRR.
We initially used the finance-elixir to compute the XIRR. One of the major advantages of this library was that XIRR computations happened in parallel. Due to this, we were able to reduce our computation time manyfold.
However, it was implemented using Bisection Method. As mentioned earlier, this method has its set of drawbacks.
The Solution
ExXirr is based on finance-elixir but uses Newton-Raphson method under the hood.
There are a few key parts of the library that do much of the heavy lifting:
The function xirr_reduction computes the xirr for the given amount(i.e value) using the formula mentioned above. The fraction variable helps represent the exponent part of the equation i.e the difference in the dates of investment.
The dxirr_reduction computes the derivation of the function. We can obtain the of the formula using chain rule which has been explained here.
reduce_date_values helps create the required structure to process the data and compute the calculated xirr value and the derivate for the given guess rate. The pipe operator |> is used to send the output of an operation as an input to the next operation.
The pmap function is the secret sauce to our speed. It helps us create parallel processes which increase the speed of computation. We can send messages to a process with send/2 and receive them with receive/1 . They are explained in detail here.
In elixir, we use the spawn to create a new process.
iex> pid = spawn fn -> raise("Moar speed") end
[error] Process #PID<0.94.0> raised an exception
** (RuntimeError) Moar speed
(stdlib) erl_eval.erl:668: :erl_eval.do_apply/6nil
In the above snippet, the new process executes our function and returns the pid of child process. However, you will notice that even though the child process has failed and exited, the parent process is still running. If we want to propagate the results of one process to another to another, we need to use spawn_link .
Our implementation of the algorithm in all it’s glory:
And here are the benchmarks.
We clearly found a 7x gain when compared to the results of finance-elixir. Switching to Elixir was a clear choice bolstered by the improved Newton-Raphson approach.
Conclusion
We are huge fans of Elixir at Scripbox as we have seen benefits with computations at scale. The power of parallelism combined with Newton-Raphson approach has convinced us to shift similar such pieces to Elixir land.
We are unlocking engineering challenges, the kind that come frequently, and with scale. If you think these are the kind of challenges that you would love to solve, do check out our open positions.
