How we use Typescript and Apache Thrift to ensure type safety
Michael Bell shares how his team uses Typescript and Apache Thrift to ensure type safety when building their latest analytics library.
This story was originally posted in our main blog. To see it follow here.
The Partner Tech mission team works closely with our business development and partner support organizations to design and implement services and tools which support publishers, airlines, and travel agencies that grow their business with us. Michael Bell, senior full-stack engineer, shares how his team uses Typescript and Apache Thrift to ensure type safety when building their latest analytics library. Their work gives our publishing partners full visibility on the performance of our inventory on their platform.
Understanding the Partner Tech team
In Partner Tech, we display our inventory — you may know them as tours and activities — on publishing partner websites. By making our bookings readily available via travel blogs, airlines, travel agencies, and publishers, we’re connecting with the greater travel community. It’s a win-win for both us and our partners. They can earn commissions from bookings, and we connect our products to people who are excited about traveling.
To be successful, our partners need insights regarding which integrations on their website are performing well. This has a direct impact on the click-through rate and ultimately revenue for them. Our Partner Tech team supports them by building tools to show the performance of our inventory on their platform.
Publishers can see each integration on our partner portal. Each integration has the first date inventory was seen, visits, clicks and the conversion rate along with its status. The team also creates interactive data visualizations that show our partner’s performance over time.
Furthermore, we aggregate anonymized data and create data models to help our partners understand optimizations related to the number of integrations, their position on the page, and geographic specific insights around their content.
You might also be interested in: How to replace type methods in Swift to improve testability
Developing an analytics package and ensuring type safety
As part of an effort to provide this granular, insightful, and meaningful knowledge to our partners, we developed an analytics package that surfaces information about our partners’ website and GetYourGuide integrations. When we started this project, we had a few essential requirements that we considered mandatory:
- Derive Typescript types directly from analytics pipeline definitions, which are currently written in Apache Thrift.
- The library had to be small and have no negative impact on our partners’ website. (At the moment it is 4kb gzipped).
- Served via a content delivery network (CDN)
- Quick iterative development
We bootstrapped the typescript library with a great open-source tool, called TSDX. If you’re new to it, GitHub describes it as, “a zero-config CLI that helps you develop, test, and publish modern TypeScript packages with ease.”
TSDX provided us with basic scaffolding, live reloading, unit testing with Jest, and Rollup for building the minified files.
You might also be interested in: How we built our new modern ETL pipeline
Apache Thrift Definitions
Thrift is a powerful open-source software framework, but for this library, we are only really interested in the code generation that comes bundled with Thrift.
Above is an example of a thrift definition we defined to collect performance information gathered from the browser.
If you want to do this too, the first thing you are going to need is to install Thrift. You should then have the thrift command available in your terminal. Now you can simply run:
This command will generate a javascript and typescript file based on our thrift definition file.
Wunderbar. You have just completed the first requirement.
At GetYourGuide, we have a rule that all Thrift definitions should be backward compatible, meaning you can only add fields, never remove. This has served us well as it coerces developers into being practical and cautious when creating/updating definitions.
Now that our Typescript definitions are generated, we can be confident during development that the types conform to our thrift definitions, and our library’s data flow is type safe. In the example below, the as keyword is a Type Assertion in TypeScript, which tells the compiler to consider the object as another type than the type the compiler infers the object to be.
You might also be interested in: 3 mindset shifts to customer-obsessed engineering in Inventory
Performance and impact
Since TSDX uses RollupJS for module bundling, we use ES Modules to structure our application and provide optimized tree shaking when bundling the library. We didn’t include any external dependencies, both for security and size performance constraints.
We ask partners to embed the script into their website manually or automatically via Google Tag Manager. Using the defer attribute, the file gets downloaded asynchronously but executed only when the document parsing is completed. This ensures the partner’s website isn’t negatively impacted or slowed down in any way.
When sending the analytics request, we didn’t want to cause a negative user experience by executing a blocking XHR request. Google Chrome 80 also disallows synchronous XHR during page dismissal when the page is being navigated away from or closed by the user.
The first step is setting up an event listener that listens to the appropriate event based on desktop / mobile events. We then use fetch with the keepAlive flag to ensure the request can be sent while the page is unloading.
We experimented using the modern navigator.sendBeacon() but found issues across various browsers, so opted for using fetch with an XHR fallback for older browsers. When the API matures in the future, I envision using the sendBeacon method.
Content delivery and iterative development
The final step was deploying the assets to a CDN, this ensures the file is replicated to edge locations around the world, meaning people even from Australia (my country of birth) download the file quickly and effortlessly.
During our Continuous Integration (CI) pipeline, we execute all our unit tests and automatically deploy the new assets to our CDN with a new unique hash, this means all user requests will automatically receive the new assets, invalidating the older assets.
This entire process allows developers to be confident their types are safe, new features do not negatively impact the library or partners, and quick iterations can be added if and when required.
Types provide a safety net for developers. When you have developers writing code all day, five days a week, your code base can expand to hundreds of thousands of lines of code. Since a code base is a constantly evolving digital organism it becomes harder and harder to create a visual map of how everything is interconnected.
When a new feature is being worked on, it may inadvertently require the developer to change functions written by others. Types provide developers a way to be explicit about the structure of their data or arguments being passed to a method. This directly translates to less bugs, less stress and confidence that your code is future proofed.
For publishing partners who have embedded the library, they can see detailed analytics around how their GetYourGuide integrations are performing along with suggestions and improvements in real-time.
We are also working on allowing partners to dynamically adjust the integrations from the partner portal. We can then use aggregated anonymous data to provide partners with integration changes from our recommendation engine. We are excited to see what our partners can do with these new tools and which insights will be unlocked.
For updates on our open positions, check out our Career page.
