Dev Diary: Using Typescript to Confidently Code Serverless REST API Products
Welcome to another Eximchain Dev Diary! Last month we released DappBot, an easy tool to turn a smart contract into a dapp in less than five minutes. We want to empower our users to create their own products on top of blockchain, so this month we are releasing
@eximchain/dappbot-api-client, our open-source client written in Typescript. That client makes it dead simple to create your own application which leverages our dapp generation capabilities, and you can learn more about it here. In this dev diary, however, I want to tell you about
@eximchain/dappbot-types, the shared types which sit at the heart of DappBot, and our key design decisions when building them.
Why Did We Build This?
We have organized our DappBot serverless code as microservices, many Lambda functions connected through a REST API Gateway. We use Terraform to manage all of our infrastructure, so our code organization isn’t defined by the popular
serverless framework. The two competing approaches here are “monorepo” vs “multi-repo”. Yan Cui broke down the tradeoffs between the two in an excellent blog post, but to make a long story short, we prefer to split the different DappBot services off into different repositories. It’s easier to separate concerns and prevents different parts of the system from becoming unnecessarily coupled. As the project grew from prototype to product, however, this strategy did cause us some development pain.
“There has to be a better way…”
— Me, writing the same data type guard in the 6th repo.
Looking across the system as a whole, DappBot’s source code currently includes:
- 10+ Lambda functions sourced from 4 different repositories
- 2 different
- An extensive Terraform config which pulls everything together with CI/CD
When we wanted to add a new feature which required adding some data to a Dapp’s definition, you needed to have an understanding of the entire system in order to make changes reliably. Each repository had its own copies of key data interfaces, like the shape of a Dapp or a User. If you didn’t know what was in each of those codebases, you couldn’t confidently make updates to the values which are shared across them. Even when the values matched, this duplication caused small headaches like the same interface being named “DappItem” on the server but “DappData” on the client.
What Were Our Benefits?
Once we had our first public release out, we decided to pay down some tech debt and create
@eximchain/dappbot-types, a public npm package which contains interfaces, constants, type guards, and factory functions for all the data shared across our system. If it moves through our API, its shape is described in that package. Centralizing these definitions made our code DRYer (more than 2.5K lines of code removed system-wide!), but it had other benefits too:
- Free from Bugs: Having one shared set of type annotations means the compiler can verify that our API is returning the exact same shape we expect to consume on the client, freeing us from an entire set of accidental “mismatch” bugs during development.
- Easy to Understand: Deduplicating these interfaces means that all data types have the same name wherever they’re referenced in the system. Enforcing these naming conventions across all the repositories makes it easier for developers to jump in someplace new.
- Ready for Change: This is my favorite part. Now when we want to upgrade a piece of the system, we just update the shared types and publish them — following semver, of course. When we
npm i @eximchain/dappbot-typesin our dependent repos, the compiler will tell us where the code no longer matches the type definitions. Fix any errors, implement new features, do an acceptance test, and we’re good to go!
How Did We Do It?
We had three guiding principles in mind as we designed this type system: safety, aesthetics, and ergonomics. Our type system ought to make it easy to write code which reads clearly and works correctly. These principles guided a few key design and implementation decisions, which are best demonstrated with some code samples adapted straight from the source.
Safety: Type Guards
One of the most dangerous issues before was the duplication of validators; different validators for the same type had different implementations, potentially leading to bugs. Typescript provides compile-time validation in the form of “type guards”, a function which accepts an arbitrary value and tells the compiler that if it returns true at runtime, the value is of the guarded type:
We used this opportunity to make sure every single one of our key types had type guards, simplifying our control flow logic across the system. Code which used to directly check for the presence of properties now instead uses these clean methods which improve feedback during development and are guaranteed to behave correctly at runtime. Here is an example using a different type guard, one which verifies that a value matches the arguments for creating a dapp:
Using these functions also makes our codebase more ready for change; if we decided to update the arguments for creating a dapp in the future, we only need to update this type guard and all of its related control flow logic will be updated.
Safety: Factory Functions
We also provided factory functions so that consumers can easily get a sample value of any type, convenient for things like initial UI states:
If you had this sample open in VS Code, the
setLoginArgs() function from
React.useState() will show that it has correctly inferred its generic type: it can only be called with
Our second focus was ensuring that our types help users write clean, readable code. “Stuttering”, an anti-pattern where part of a name gets duplicated over and over again, was one of the top problems we wanted to eliminate:
Repeating the words
Method increase the amount of “noise” in the expression, extra words which don’t add more meaning. However, if you have multiple argument types defined in the same file, it can seem like this stuttering is the only way to ensure names don’t collide. Here we took a leaf out of Python’s book:
Namespaces are one honking great idea -- let's do more of those!
— Tim Peters, “The Zen of Python”
Typescript’s namespaces let you collect a set of names and separate them from the others, preventing collisions. We leverage this, for instance, to create a helper collection of types from Stripe’s API:
While Stripe’s API has numerous types nested across a large structure, our clients will only ever interact with the small subset shown above. Using this namespace, we can collapse some of the stuttered nesting (e.g.
subscriptions.ISubscription) and provide an easy import for our consumers:
Aesthetics: Naming Conventions
We also wanted to ensure that we use the same naming conventions across the API. For any data type named
[SomeData] in our API, its corresponding type guard and factory signatures look like:
This consistency makes it easy to work across the API, as you always know where to reach for a given method. Want to verify that an API call was successful, or that an object is a valid Dapp? Use
Item.isFull(). Need a blank Dapp object?
Item.newFull() is ready to go.
These names are supported with heavy docstring usage throughout the package. While we strive for our internal variable names to be self-explanatory, this package is also intended for outside consumers who have not used our API before. Adding docstrings to the types sitting at the core ensures that they will carry through to all consuming libraries. For instance, this GIF shows all of the mouseover documentation available on our authentication challenge data types when used in the :
Ergonomics: Standalone API Method Imports
Last but not least, we focused on making sure the “code ergonomics” supported an easy developer experience. For instance, we wanted to guarantee that you could get all of an API method’s types from a single import. We used namespaces again in order to provide a consistent, compact import for every one of our methods.
Methods are first grouped by the root API resources they belong to, then each method contains all of its related types and constants with consistent names:
Grouping the path, method, arguments, and result for each method in one place makes it easy for an API consumer to get up and running. Simply import the types for the method you’re using and all of the required data is immediately available.
Two keys pieces of this to note:
Pathvalue depends on two externally defined constants:
authBasePathis the shared root resource values, (i.e.
ResourcePathscontains all of the string path values for this resource (i.e.
/v1/auth/login). If we need to change the shape of the API in the future, we update the constants in the types package and everything else will automatically reflect those changes.
- We separate the
Responsetypes because our API uses a standard shape for all of its responses, described earlier in this post (i.e. every response has both
Resulttype describes the contents of
datain a successful responses. Defining these types separately lets us easily type both the server-side implementations (i.e. the method’s handler must result a
Resultto the main API function) and client-side handlers (i.e. the client will receive a
Responsewhich hopefully contains the
Here is a snippet showing how you could use these types with the
Ergonomics: Module Layout for Flexible Imports
The last key design decision was how we laid out the underlying file structure. When we decided to group all of our types into one package, we didn’t want consumers to have to write out long dot references in their code. Instead, we organized the types in nested directories such that our endusers could import names from a more specific module:
In order to support this syntax, we needed a module layout which looks like:
Each of these submodules contains three files which follow the same pattern in the
spec/user submodule. The
user/user.ts file is where all of the actual
User types are declared and exported at the top level. The
package.json files all follow the pattern shown below:
package.json was one of the last things we had to figure out! Our namespaces export a mix of types (e.g. the
Login.Args interface) and values (e.g. the
Login.Path string). We want Typescript to find the types at compile time, then we want node to find the values at runtime. This file gets the job done! The
types key tells Typescript that the adjacent file contains type definitions, while node will follow the
main key to find the actual compiled output.
Note that if we only wanted users to be able to import a top-level
Types object, then we wouldn’t need these nested files — the top-level
package.json would have all the info required. However, when we further specify an import folder (e.g.
@eximchain/dappbot-types/spec/user), node is now treating that specific folder as an independent module. Including the additional
package.json makes sure that behaves correctly.
“Where Can I Try It Out?”
We hope that this blog post shines some light for future developers who are contemplating writing their own serverless REST API in Typescript! Its compile-time checks make it possible to enforce guarantees across all of your interrelated codebases, letting you code with confidence.
Both of these packages source code is available on GitHub, as well! If you’re in the process of building your own serverless API product, the dappbot-types source on GitHub could be a source of inspiration for clean, scalable patterns.
Eximchain enables businesses to connect, transact, and share information more efficiently and securely through our blockchain utility infrastructure. Using Eximchain blockchain technology, enterprises can eliminate traditional supply chain barriers and integrates actors big and small into an efficient, transparent, and secure global network.