Introducing the US Pricing API: Our Evolution from a Static Model to a Dynamic Interface
Have you ever wondered how we manage the complexity of pricing data at our organisation, and how this system has evolved over the years to meet our ever-changing business needs?
For years, our suite of services has relied on the battle-tested US Pricing Model. This robust and reliable model was designed to encapsulate our pricing data efficiently and effectively in the form of JSON files, a popular, lightweight data interchange format. By using JSON we have been able to ensure our pricing data is easily readable and accessible. This transparency has played a crucial role in our operations, allowing for clear and consistent pricing data across our entire platform. The JSON files are versioned and committed to the repository, offering us the ability to track changes over time and maintain a historical record of our pricing data.
While this system has served us well, the rapidly evolving technology landscape and shifting business needs have prompted us to reevaluate our approach. Staying still is not an option and in fact, our ability to adapt and evolve is what sets us apart. With business needs growing in complexity, and with an eye towards scalability and dynamism, it became clear that it was the perfect moment to revamp our pricing system.
The Challenges With The US Pricing Model
Our current pricing model has proven itself to be reliable and effective over the years. But, as you’d expect, as we’ve moved forward and our needs have evolved, we have identified challenges that we needed to overcome.
In particular our pricing information has been tightly coupled with our repository.
This has meant that any adjustments or updates to the pricing data necessitate a new deployment. While this structure has its merits, the downside is that it can lead to slower response times when it comes to reacting to market dynamics.
This inflexible setup also meant that stakeholders lacked the ability to self-service changes to the pricing configurations. Every modification, regardless of its size or complexity, required the involvement of the engineering team. This process not only added to the workload of our engineers but also led to the additional overhead of communications. Ensuring all teams and stakeholders are aligned and that prices are well validated before they affect borrowers became a complex task.
Furthermore, the rigidity of our existing model had the potential to hinder innovation and experimentation, two critical elements in today’s rapidly evolving marketplace.
In recognizing some of the challenges we were facing, we understood the need for a change in our approach. The transition from the current US Pricing Model to the more dynamic and scalable US Pricing API was not just about overcoming these obstacles, but also about fostering an environment that encourages innovation, quick response to market changes, and efficient communication.
The Design Phase: Mapping the Way Forward
In response, we therefore initiated the US Pricing API project. This new endeavour was conceived to address the existing challenges head-on, and the design phase served as our roadmap to achieving this objective. We focused on implementing a series of key enhancements aimed at driving efficiency, improving response time, and fostering innovation.
One of the major advancements we sought to introduce was the storage of pricing information in a database. This strategic move away from repository-bound JSON files was geared towards facilitating quicker and more efficient updates. By storing pricing information in a database, we could more readily accommodate changes without the need for a new deployment, thus enhancing our ability to respond to market dynamics in real-time.
Another significant aspect of our design phase was the development of an API to manage pricing information. This was intended to streamline the process of updating and maintaining our pricing data, thereby reducing the involvement of our engineering team in everyday pricing adjustments and modifications. With this shift, we aimed to empower stakeholders, allowing them more control over pricing configurations and decreasing the dependency on the engineering team.
In addition to these changes, we also planned for the creation of a user interface. A dedicated UI would make the pricing data more accessible and easier to manage. It would allow non-technical stakeholders to oversee and administer changes, thus promoting a more collaborative and inclusive environment.
Furthermore, we identified a need for refactoring the communication between services. Simplifying these communication lines aimed to minimise the risk of potential errors and misunderstandings, resulting in smoother and more efficient interactions between different services.
Guiding us through this design phase were best engineering practices. We remained committed to transparency and collaboration, documenting the decision-making process in GitHub. This not only provided a record of our thoughts and choices but also served as a valuable reference for future decision-making.
We also made use of Architectural Decision Records (ADRs), an effective tool for capturing important architectural decisions along with their context and consequences. These records provided us with a concise and clear history of decisions, offering insights into the why and how of our choices.
Carrying out a trade-off analysis was another vital aspect of our approach. It helped us to weigh the advantages and disadvantages of different options, ensuring informed decision-making at each step of the design process.
Perhaps most importantly, we fostered active participation from a diverse team of engineers throughout the design phase. This collaborative environment was instrumental in bringing a variety of perspectives to the table, leading to more robust and well-rounded solutions.
In essence, the design phase was a critical juncture in our journey from the US Pricing Model to the US Pricing API. It was here that we charted the course for our new and improved pricing system, and we did so with careful consideration, strategic planning, and active collaboration.
The Implementation Phase: From Design to Action
As we shifted from the design to the implementation phase of the US Pricing API project, our primary focus was to convert the static US Pricing Model into a more dynamic and responsive Pricing API. This transformation was marked by a series of carefully executed steps, each addressing a unique aspect of the transition, all while adhering to best development practices.
We began this journey by developing a new service specifically designed to host the Pricing API. The selection of Ruby on Rails as our development framework of choice was not coincidental. Known for its developer-friendly syntax and feature-rich ecosystem, Ruby on Rails is a robust tool perfectly suited for building scalable and maintainable web applications.
As we implemented the Pricing API, we were keen to maintain the integrity and stability of our systems. To ensure backward compatibility and smooth transitions between API versions, we adopted a versioning strategy. This approach allows us to roll out new features or modifications without disrupting existing services that depended on the API.
Documenting our work was a top priority throughout the implementation phase. Detailed documentation provides vital insights into the system, facilitating future troubleshooting and fostering a better understanding of the system’s workings. It essentially serves as a reference guide for both current and future developers, ensuring consistency and efficiency in further developments.
To enhance the robustness and reliability of the Pricing API, we implemented rigorous input validation and error handling mechanisms. Input validation safeguards our system from potentially harmful data, while robust error handling ensures that the system can respond gracefully in the event of unexpected issues, providing meaningful error messages to guide users and developers.
Knowing that the backbone of any reliable application is its ability to handle extensive testing, we focused heavily on this aspect. We engaged in rigorous testing, ranging from unit tests to integration and end to end tests, to ensure our API worked as expected and interacted seamlessly with other services.
Moreover, to gain insights into our API’s performance and spot potential issues before they escalate, we implemented logging and monitoring mechanisms. Logs provide a real-time record of events, offering valuable debugging information. Monitoring, on the other hand, helps us keep track of the system’s health and performance, enabling proactive maintenance and prompt issue resolution.
In summary, the implementation phase marked a critical period of turning our designs and plans into tangible results. By adopting these best practices and maintaining a meticulous approach, we ensured the delivery of a reliable, efficient, and highly scalable Pricing API. This step not only transformed our pricing system but also created a strong foundation for future innovations and improvements.
Testing: Ensuring Reliability and Consistency
The assurance of system reliability and consistency is of paramount importance, especially when implementing changes to core components like our pricing system. Our testing approach, therefore, was carefully planned and executed to ensure the successful transition from the old US Pricing Model to the newly developed Pricing API.
Our primary testing strategy was to operate the new Pricing API concurrently with the older US Pricing Model. By running these two systems in parallel, we could directly compare the output from both, ensuring the new API was delivering consistent and accurate results. This ‘shadow testing’ approach allowed us to verify that the Pricing API was functioning as intended without disrupting our existing operations.
We employed robust comparison methods to scrutinise the results generated by both systems. If any discrepancies surfaced, we thoroughly analysed the case to understand the root cause. Be it a flaw in the new implementation or an unanticipated edge case not covered by our tests, our aim was to ensure complete parity between the two systems’ outcomes.
During this parallel operation phase, we were also able to examine the new system’s performance under real-world load conditions, giving us a chance to observe and optimise any performance issues. It also gave us valuable insights into how the Pricing API would interact with other components of our service architecture, thus eliminating any unforeseen integration challenges.
After running the two systems side-by-side for a substantial period without encountering any discrepancies or errors, we felt confident about the new system’s stability and readiness. Only then did we proceed to decommission the old US Pricing Model and made a complete transition to the new Pricing API.
This testing phase was not just about finding and fixing bugs; it was a crucial period of validation and fine-tuning. It provided us with the confidence that our new Pricing API was not only functionally equivalent to the old system but also poised to bring the enhancements and scalability we aimed for. The meticulous testing approach was our safeguard to ensure a smooth transition and the continuous reliability of our pricing system, a critical component of our services.
Reflecting on our transformation from the static US Pricing Model to a dynamic Pricing API, I appreciate how we anticipated and tackled scalability and dynamism challenges. This was no ordinary task; it required innovative thinking and quick adaptability to foster a new environment. Today, we’re better positioned for growth, paving the way for future improvements.
Finally, It’s important to note that software engineering involves much more than simply writing code. It’s a craft that requires a broad range of skills and knowledge, from project management and problem-solving to testing and collaboration.
