A review of 2019
What a ride it has been! 2019 has been a truly exciting year for all of us at StashAway! We have finally said “good bye” to WeWork in Singapore and moved into our own office in December, we have welcomed our 50th employee during the year, we have launched a number of exciting investment products, while continuing to push to provide the best possible experience for our customers and scaling the business to new dimensions!
But let’s look at our accomplishments one-by-one:
New products and features
As a customer these would be things that you already know — but here’s a brief recap of everything that has been launched over the past 12 months:
Global portfolios
In order to respond better to changes in the global economy we have expanded the universe of ETFs that we invest into and upgraded our core portfolios to what we now call our “Global portfolios”. You can truly see their global nature by looking at the distribution of underlying assets across the world map — we have also introduced some nice new portfolio details together with the rollout of this new set of portfolios.
MyInfo onboarding
Unless you onboarded as a new customer in the past months you wouldn’t be aware of this one — onboarding for Singaporeans now became even easier by having the option to sign-up using MyInfo, where we can use government-verified data to verify your identity and thus skip some steps in our normal account-opening process. This makes the experience of becoming a StashAway customer even smoother and faster than it had been before!
Income portfolios
Due to popular demand from our customer base we’ve decided to launch a new set of portfolios, traded exclusively on the Singapore stock exchange (SGX), comprising a set of asset classes that are optimised to generate dividend yield that can be used to construct an “income generating” portfolio — ie. you put down a lump sum and use the dividends (which are paid out automatically each month) to cover your living expenses. Of course, you can also choose to reinvest dividends as it’s always been the case.
Intergoal transfer
With the larger number of different investment portfolios available, the demand to transfer assets between these different portfolios has been ever-increasing. It is now possible to initiate a transfer of a certain (or the entire) amount of a portfolio into another one. The system will determine the optimal currency for you automatically, to incur the least amount of FX conversion fees as possible.
WhatsApp communication
We have been working closely with Facebook (which owns WhatsApp) since the beginning of the year to bring a better communication experience to our customers. We were one of the first businesses to open a dedicated business account in Singapore, and to use their APIs to facilitate the communication and integration with our existing customer service desk software, Zendesk. We are using a tool called Zendesk Sunshine (formerly “Smooth”) to integrate the WhatsApp API into our existing workflows.
StashAway Simple
With one of the most successful product launches since the inception of StashAway, you can now also manage your cash with us! StashAway Simple offers a no-frills low-risk portfolio that you can invest your cash into as an alternative of having it sit in a no/low interest bearing savings account with your bank. Our value proposition of keeping things “simple” as well as passing on cost savings to our customers are the key drivers behind the success of this latest product.
Design upgrades
Aside from the above mentioned features we have also continuously improved the look&feel of our mobile and web applications. Behind the scenes our product managers and designers are operating a comprehensive design system that ensures a consistent experience across the different parts of our product. Here are some examples of before/after comparisons of some of the redesigns that we have rolled out in the past months. It demonstrates nicely the amazing progress we have made in the last year!
Backend automation
A lot (if not most) of the work of our engineering team has actually happened behind the scenes — that is, invisible to our customers but extremely important to ensure the scalability of our systems and processes. Automation is at the core of our business model, as we would not be able to sustain our low fee structure if operations were to require as much manual personnel as a bank requires, for example.
A big shoutout therefore here for everyone who doesn’t work on shiny new features but instead helped to improve our daily backend processes bit by bit, day by day. The following are just some of the more interesting examples of the many things that we have implemented over the past 12 months.
Citibank API integration
If you think that banks provide nice and easy-to-consume RESTful APIs with great documentation — think again. Many of the underlying core banking systems are quite archaic and difficult to maintain. APIs therefore often times are constructed around the processes that these systems enforce, rather than having intuitive APIs and well-formed inputs and outputs that you can rely on. Nevertheless, in order to reduce human error and to make our ops-team members’ lives easier, we have integrated with Citibank to automate the sending of cash transfers via their API, instead of having to rely on manual entry. As this is obviously a crucial and sensitive functionality of our system we have established maker/checker processes using the bank’s existing interfaces to enforce additional independent checks on any monies leaving our bank account.
The “1 cent problem”
Talking about archaic core-banking systems… our broker uses two different sets of data when communicating cash transfer and trade executions to our systems. One, more real-time, but less accurate; the other, slow, but the actual data used in the accounting systems. The real-time system unfortunately works with a different precision on the number of digits used when rounding prices of securities or FX rates compared to the batch-style core banking system (9 vs 13 decimals). On some days this can lead to a mismatch after rounding of 1 cent. Now why do we care about this cent when we are converting and transferring millions of dollars, you may ask? It’s because we are reconciling our systems with all external parties on a daily basis, and any discrepancy will be flagged at this point. This is important to ensure that we detect potential issues right away, and even a small difference could become problematic in case we are for example paying out an amount of money to the customer that we do not have. Since customers’ assets are strictly segregated from the company’s assets we also cannot simply take a cent from the company’s cash balance to make up for the difference. Instead, we have implemented a correction flow that would take the spare (or missing) cent and adjust the internal transfers accordingly to ensure that numbers match up.
This level of accuracy may seem unnecessary, but in fact it’s a key principle of our automated systems — being able to operate with 100% accuracy, 100% of the time. Any (even minor) mismatch causes manual effort to investigate the root cause which we want to avoid.
Cadence
By now you have probably realised how much we care about processes and their predictive and reliable execution. When operating in a microservice environment like ours, the sheer number of services involved in some of our processes can be daunting and difficult to follow/understand. This complexity can become problematic over time, as it will surely slow down the speed at which the engineering team can develop new features, rollout new countries, operate on scale, etc.
Therefore, in the middle of the year we started looking into systems that would help us manage our workflows by orchestrating the various microservices that are involved in the workflow. Workflows could also be long-running (eg. a withdrawal can take a few days to complete). After careful consideration we decided to give Uber’s Cadence a try. It uses Cassandra to manage workflow state and utilises the concept of event sourcing, both technologies that our engineers have already gathered experience with in production with our existing services. Cadence allows workflows to be expressed in code, and abstracts away the potentially asynchronous nature of interactions with other services by allowing workflows to wait for signals from other services. For example, during onboarding of a customer you could have a workflow that checks the eligibility for some sort of discount if a deposit arrives within the first 30 days — this can literally be expressed as “sleep 30 days” and use a signal in case a deposit arrives to interrupt the sleep and process the discount.
We have started migrating our first workflows to Cadence and are planning to use it much more in 2020.
Infrastructure
Another big focus of our attention in 2019 has been the improvements to our infrastructure which is running in AWS on Kubernetes. Our infrastructure team ensures that our platform is reliable, secure, and scaling well, and is supporting other engineering teams with launching of new services.
APM for tracing requests
For those of you who don’t know, the ELK stack (Elasticsearch, Logstash, Kibana) has an application performance monitoring (APM) tool that allows tracing of requests across microservices. This comes in handy during the debugging of slow endpoints or degraded application performance.
We have already put this to good use and improved the performance of several endpoints by analysing the traces as shown in the chart above.
Elasticsearch
Elasticsearch again, but this time not for logging but as an actual application database. In the Admin interface that our ops and client engagement teams are using to handle customer related transactions and enquiries we have to search for customers by their name, email, or phone number in certain places — for example when a customer calls in the call center, or when we need to search customers for the mapping of a money transfer that came in without a proper reference code. Due to the large number of customers in our system this search had become very slow over time to the point where it started crashing some of our internal services that were responsible for handling internal search requests (mostly due to out of memory errors).
We have subsequently spun up an Elasticsearch instance which is hooked up using Monstache to our core Mongo DB and therefore has an updated index of our customer data. We can then perform complex searches against that index which performs in orders of magnitudes better vs. the initial naive approach of walking through the entire list of customers (our search queries are non-trivial, thus we cannot just do with a standard DB text query).
Disaster recovery testing
Last but not least, in 2019 we performed the first disaster recovery test on our systems. During a disaster recovery test you typically simulate the outage of one or multiple system components to ensure that the mechanisms in place to handle fail-overs and recover from a system outage work as intended.
In our case we simulated the outage of an entire AWS data center in Singapore by simply cutting the network connections between AZ 1/2 to AZ 3. Kubernetes will automatically reschedule stateless services into the remaining AZs, but stateful services (eg. database systems) might be a bit harder to recover. In our case we are using MongoDB, PostgreSQL, and Cassandra as our production database systems which are all running in cluster mode and can handle the outage of an entire data center (ie a third of the available cluster nodes).
As you can see, the technical challenges that we are facing are becoming more and more interesting as the company grows in terms of scale (number of customers and transactions), breadth (number of investment products), and complexity (number of countries, number of engineers, number of services).
We are super excited for the coming year and the new challenges that are awaiting us! It’s not gonna be easy — but then, if it was easy, it would already have been done :)
If you are up for a new challenge in 2020 please check out our careers page and feel free to reach out to us directly — we are looking for capable engineers across most of our teams in all seniorities.
