The challenges of software development at a rapidly growing fintech — Part II of II

Caio Abe
Caio Abe
Dec 17, 2019 · 9 min read
Illustration of a tape measure with the SOLID acronym written on its side

In the first post of this series, I discussed the birth of Creditas’ Servicing Tribe and what it was like to develop a new context in an application that was already in production.

Reading the first post is essential to understanding the context of this post, so if you haven’t read it yet, you can start here.

I mentioned in my first post that, after the Calculator had been created, we faced the challenge of extracting it to a new service in order to pass the torch on to the Pricing team, whose scope is to unify all the calculations which emerged throughout the company into a single place.

I split this post into two parts:

Part I:

  • Historic context
  • Servicing Map context
  • Challenges and opportunities of a new context
  • The development process
  • Other sailors on the horizon
  • Conclusion

Part II:

  • Extraction strategy (Branch by Abstraction)
  • Success factors for extraction
  • SOLID applied at the architectural level
  • Conclusion

Integration with the Pricing team

The main languages we considered were Scala (for its Functional First nature), Python (for its mathematical libs), and Kotlin (for its ecosystem and compatibility with other movements that already existed inside Creditas).

Fortunately, we decided to be pragmatic. We postponed the decision to leave Ruby to time with more business certainties and a better idea of the direction the company would take with regards to these new stacks.

Considering that, at the time, the cost of creating a new context within the existing application was low, so we stuck with Ruby.

[GIF] Hand with decorated glove unfolds a cloth uncovering a Ruby in a heart shape

The other teams at Creditas shared this same mindset, making their decisions with a cautious and systematic vision, given that many new resolutions were appearing within the company. Thus, for the moment, the Pricing team was conceived with Ruby, which helped reduce costs (for everyone).

This gave us many comforts: there wouldn’t be a Ruby learning curve among teams, we had agility because we wouldn’t have to “translate” one language to another, and we wouldn’t have infrastructure issues since we already possessed expertise in provisioning a stable production environment in Ruby.

Only alignments between the business and the structure of our project were necessary in order to pass the (context) torch to Pricing.

Success factors for extraction

This quote by our Rafael Manzoni, the Tech Leader who led our team’s deliveries, has my approval:

I think that, yes, SOLID has given us dynamism and decoupling in the calculations, making it easier to configure each component. Yet the success of the extraction, in my opinion, was due to the joint use of strategic and tactical DDD patterns, such as the Context Map and the Integration between contexts using ACLs (Anti-Corruption Layers), which facilitated everything.

Sequence diagram that exemplifies the communication between contexts via API in a simplified manner

Note that if we wanted to change the Inter-Process Communication strategy, being it by internal controllers, HTTP requests or any other protocol, we could just insert a different implementation of the Gateway without much effort.

I believe that the successful creation of the decoupled context and its ease of extraction can be attributed to the aforementioned factors, but I think there’s another special ingredient which hasn’t been mentioned yet:

A team that — above personal interests — plays to win as a whole, not only thinking about the technical solution for the Squad or the Tribe, focusing instead on solutions for the entire company.

Here is a list reflecting upon some of the challenges we had on this journey:

  • How will the Billing context access the Calculator with minimal coupling?
  • Where is the limit when it comes to creating excessive complexity between these integrations? Should these decisions be postponed due to the possible technical debts or advance them at the risk of possibly designing these abstractions incorrectly?
  • How would the integration tests be done in the layers of the Billing application? Reliability with duplication versus confidence with DRY?
  • How will the design of the Calculator’s Use Cases work, given that it can become an API in the future?
  • What on earth is Integration between Bounded Contexts, and why won’t anyone let me code in peace?

All jokes aside, it was thanks to the know-how of some very experienced developers, many debates, and an exploration of DDD Study Groups that enabled us to learn and apply everything the way we did.

Are you interested in these challenges and study groups? Let’s get to know each other.

Extraction strategy (Branch by Abstraction)

[GIF] Meme of a poor raccoon who is disappointed when discovering how a block of sugar behaves on land compared to a new environment (water)

We had to address Reliability first since the service was new and we had to ensure the calculations wouldn’t fail during our daily processes. Therefore we decided to proceed with a Branch by Abstraction strategy, where we basically replaced the calculator’s call with an abstraction that allowed us to use the new flow and the old flow, if the former were to fail. Basically, all the classes that use the Calculator at the application layer of our codebase came to use one service and two gateways:

  • External gateway, which realizes the calculation with an HTTP request
  • Internal gateway, which calls the controller from the local copy of the calculator

The way that our Fallback component is used, which realizes switches between our legacy flow and our new flow, looks something like this:

# service.rb@fallback_wrapper.new(main: @http_calculator_gateway,fallback: @internal_calculator_gateway,error_event_name: ‘installment_calculation_failed’).perform(dto)

The Fallback component sends a perform message to the HTTP gateway, passing a DTO (Data Transfer Object) as a parameter for both. If the principal flow fails, the alternative internal gateway flow is used, and the error message ‘installment_calculation_failed’ is logged in Transactional Events in New Relic.

This strategy ensures proper function using both flows and helps us guarantee our service’s reliability.

We then had to deal with Latency since we now had to perform an HTTP call for each installment. The problem was that some use cases calculated 180 installments at a time. Our solution was to abstract the gateways and services so they made batch calls. We lowered our latency from minutes to mere seconds.

Our Billing context continues using the internal Calculator as a Fallback in some cases, but our monitoring of these errors in APM has already shown us that these parts of the flow will soon become obsolete because of how rarely these errors are logged. In other words, we’ll soon be able to delete the legacy code and depend solely on our Pricing microservice!🎉

[GIF] Edited scene: Rafiki throws baby Simba off the cliff in the iconic Lion King movie scene

SOLID applied at the architectural level

Servicing contexts: Billing (charging), Onboarding (new credit on the platform), and Calculator (calculator)

SRP: Single Responsibility Principle

More than simply “Do one thing”, the segregation of the Calculator, corroborated by Uncle Bob, communicates something more important: “Only one reason to change”. Given that this module was isolated to a new context, the Calculator’s experimental and volatile business interests (ex: new policies every week) don’t interfere with the responsibilities of Billing and Onboarding (entrance of new credit). There are no shared responsibilities between contexts, so it was easy to work on the different fronts of the same codebase together.

Arrows pointing towards the domain

OCP: Open Closed Principle

HTTP and InternalGateway abstractions fit into the service

LSP: Liskov Substitution Principle

Hot dog stand that offers açaí, pastry, website, yakisoba, and acarajé

ISP: Interface Segregation Principle

Our first Service version had three use cases all in one Service: calculation of the components of an installment, creation of an installment plan, and a calculation of the inflation index. In order to avoid cross-module dependencies when creating strategies for flow bottlenecks — reducing complexity and mitigating the probability of risks — we segregated the interfaces into more granular pieces (three services) so we could proceed with the extraction in an incremental manner which is safer.

Three entities conversing. On top: “I orchestrate”, Left: “You do it!”, Right: “Inject me there”

DIP: Dependency Inversion Principle

Conclusion

I hope to have advocated constructive reflections and materialized this abstract world which resides in books with real examples that we’ve run into around here at Creditas.

Leave your cases in the comments so we can learn more together!

Photos of the original illustrations used as the post’s cover

As a dev-ex-designer, I still do some scribbling between codes. Of course any iterative work — including the writing of code — results in some rough drafts prior to the final version. I choose to use the illustration of SOLID on a tape measure rather than SOLID as architecture in this post to depict that despite all the architectural lectures, we use our fundamental daily tools to be excellent software masons.

Thanks for your time and until next time, folks!

Want to use technology to bring innovation to the loan market? We’re always looking for people to join our Crew!

Check out our openings here.

Creditas Tech

Our technologies, innovations, digital product management…