Upskill tutorial for separating API routing and database interactions

Day 25: Operations layer. Exceptions handling. Protocol.

Hi 👋 I am Hud, a postdoc for engineering data science at the AI Manufacturing Center in Laramie, Wyoming. My funding is running out (AaAaaA !), so while I am actively looking for a new job, instead of doing the 205th coding certificate to prove my worthiness — I thought I’d do design challenges and document how I spend my time upskilling so other engineers can do the same.

Nowadays, certificates are everywhere. Documenting small upskill projects that you can later show off is the best way to get recognition as a professional engineer.

This is day 25 of a 30-day design challenge. Follow along and let me know if you get stuck!

TL;DR tasks

Download the provided code for the challenge.

Task 1: Create an operations layer

• Introduce a separate module or package for the operations layer.
• Implement functions in the operations layer to perform database operations such as creating, retrieving, updating, and deleting events.

Task 2: Define the interface between the operations layer and API routes

• Determine the input parameters and return values for each function in the operations layer.
• Ensure that the operations layer functions handle errors and return appropriate responses.

Task 3: Refactor the API routes

• Modify the existing API route functions to map to the corresponding functions in the operations layer.
• Replace the direct database operations in the API routes with calls to the operations layer functions.

Task 4: Rewrite the unit tests for the operations layer

• Create new unit tests specifically for the operations layer functions.
• Ensure that the unit tests cover various scenarios and edge cases for each function in the operations layer.

Task 5: Test the API routes with the operations layer

• Run the API and test the functionality to ensure that the operations layer is correctly integrated with the API routes.
• Validate that the API still functions as expected after refactoring to incorporate the operations layer.

Bonus Task: Update the unit tests for the API routes

• Modify the existing unit tests for the API routes to account for the changes made with the operations layer.
• Verify that the unit tests for both the operations layer and the API routes provide good coverage and ensure the reliability of the API.

The problem

This challenge is about separating the API and database operations. Separating API and database operations is a good idea because it helps to keep the code organized and maintainable. By separating these concerns, you can make changes to one layer without affecting the other. This can make it easier to test and debug your code, and can also make it easier to scale your application. Look at this:

Both methods interacts directly with the database, but also raises a HTTPException when the API route is not available. So how can we solve the problem?

Operations layer

The operations layer can help by providing a clear separation between the API and database layers. This layer can contain functions that perform database operations, such as creating, updating, and deleting records. The API layer can then call these functions to perform the necessary operations. This can help to keep the API layer focused on handling requests and responses, while the operations layer handles the database interactions.

Additionally, we can reuse the operations layer in certain scenarios e.g. you have an in-house tool that doesn’t work with the API but interacts with the database — reusing the operations layer will reduce duplication of code.

Let’s have a look at how we can implement this:

That’s right! The good ‘ol abstraction Protocol from Challenges 3, 9 and 14. Now we can decouple the database interactions from the routing operations:

Looks nice and clean, but not for all methods — notice the NotFoundError raised in the operations layer here:

And how it’s implemented in routing:

Both NotFoundError and HTTPException are somewhat in conflict —One could potentially put all exception handling in the operations layer, but in my opinion HTTPException should really be coupled with routing. So how do we resolve this?

We can simply catch a NotFoundError and translate that into the HTTPException ! This way the exception handling for routing and database interactions are clearly separated.

Finally, one could also potentially put all of the SQLite boilerplate in the operations layer to separate it even more — but I will leave that to you on your journey!

Conclusion

Congratulations! You finished Day 25 from the 30-day design challenge.

If you have reached this far, you know how to:

• Create an operations layer with protocol to separate database interactions and API routing
• Translate exception handling for database interactions into API routing

TBU: Check out the day 26 challenge!

Also, you can access the full 30-day GitHub repository here.

💡 My goal here is to help engineering data scientists upskill in design. I’d like to hear from you! Was this helpful? Anything I can improve? Connect with me on LinkedIn | Medium