Using Cloudflare Workers to Restructure URLs
At Guild, we have an increasing number of employer partners that have their own education benefit policies. As a result, the set of supported academic programs for an employee of one company may look different than the set for an employee of another company. Until recently, regardless of which company a potential student was employed by, when she came to our catalog, she would see the same offerings, in the same order, as everyone else. This was not an ideal user experience. We realized that it would be far better for users to have a more personalized set of options to choose from.
Our solution was simple: employer-specific catalogs. In fact, with this idea as the catalyst, we saw the potential for offering a more holistic and personalized experience throughout the Guild ecosystem. However, before we could establish this new feature, there was a bit of groundwork that had to be done. It required taking a step back, looking across all of our applications, and restructuring our URL patterns to support a cohesive and personalized user experience.
The Vision
At the start of the project, we only handled partner-specific content via informational “partner pages” in our core application. These pages could be found at a URL that looked like this: www.guildeducation.com/partners/<employer>. Standalone applications, like our catalog, were available at subdomains like catalog.guildeducation.com. In order to make transitions from one application to another feel smoother, we decided to swap the path and the subdomain elements. Employer identification would be handled with the subdomain and other applications would be available via paths.
For example, in the old system, a company named ACME Inc. might have had an informational page at www.guildeducation.com/partners/acme, and an ACME Inc. employee, like any other user, would view the catalog at catalog.guildeducation.com. In the new system, the employer’s informational page would live at acme.guildeducation.com/partner and the new, tailored catalog would be found at acme.guildeducation.com/catalog.
How we did it
To make this happen, we had to make changes in a few places across our system.
- In the core application, that currently handles the partner pages, we updated the logic to render these partner pages at their respective subdomains, rather than the paths that they were previously loaded on.
- For other applications, that were (and, technically, still are) served on their own subdomains, we implemented a reverse proxy with Cloudflare Workers.
Partner Pages
The first piece, moving www.guildeducation.com/partners/acme to acme.guildeducation.com/partner, was fairly straightforward, so I won’t go into the weeds in this step. At a high level, it required creating some new DNS rules to handle each employer subdomain, changing the routing and controller logic to render the content at the appropriate URL, and adding redirects to handle any legacy links that pointed to the old URLs. Additionally, some of our partner pages have several pages nested within their primary page, so we had to handle paths like /partner/overview and /partner/college.
One interesting thing that came out of this work was the way that worked in development. We used to be able to fully interact with our local applications by simply visiting “localhost” in a browser. This met all of our needs because we served each application its own domain or subdomain. We did not previously have an application that handled multiple subdomains. With the change, we now need to be able to access something like acme.localhost.com and that just does not work. Our solution was setting up a domain that resolves to 127.0.0.1. This allows the work with subdomains in a local environment.
Reverse Proxies with Cloudflare
We had several applications that we wanted to put behind a reverse proxy, so that a user could visit www.guildeducation.com/<app-name>, and access an application that was hosted on a different domain. One that I’ve mentioned several times is our catalog. We needed to make it available at www.guildeducation.com/catalog and at subdomains like acme.guildeducation.com/catalog. To accomplish this, we decided to use Cloudflare Workers.
Cloudflare Workers are similar to service workers, only they don’t run in the users’ browsers. The are hosted on Cloudflare’s Edge network. Essentially, they are services that run JavaScript on any route that you specify in your application. We used these workers to intercept requests and return new ones. If you check out the docs, you’ll see that their example is a simple handler that responds to a fetch event and does some logging. Our code is structured the same way, only it returns a new request instead of the original. It looks something like this:
In addition to the JavaScript above, we had to explicitly tell Cloudflare which routes to execute this script on and which paths to exclude. For example, we want the script to run on www.guildeducation.com/catalog/offerings/123/program, but we don’t want to it to run on www.guildeducation.com/catalog_some_backend_route. The API makes this a relatively straightforward process. In addition to providing an interface to add the script, we specify exactly which routes we want to “enable” and which routes we want to “disable”. We had to be extremely specific when adding these routes to their respective category, although we were permitted to use wildcards at the beginning and end of the specified URLs.
We also had to make changes to the routing logic inside each of the applications that would be proxied. Namely, we had to add a basename in each of the applications’ router configurations. Paths that used to begin at the root of an application, now came after a path segment that identifies the application itself. For example, in the old model, when we accessed the catalog at www.catalog.guildeducation.com, we would find a program detail page at /offerings/123. Now, that we access the catalog at www.guildeducation.com/catalog, the program detail path is technically /catalog/offerings/123. Every path and link in the catalog application must now begin with /catalog. To accomplish this, we set the basename to “/catalog” in the router configuration.
One final gotcha with this reverse proxy setup: We use Webpack and our public URL for loading the application bundles was previously a relative path. That’s all well and good when you are loading and accessing an application from its actual domain. When we implement the reverse proxy and users access an application from a different domain than its original source, the relative path no longer works. To fix this, we had to make sure that our Webpack bundles were loaded from absolute URLs when not in a local environment.
Wrapping Up
We’ve set up employer partner subdomains and the reverse proxy in several applications so far, but we have more work to do. We are now working on proliferating the new URL structure throughout the Guild ecosystem. Even as the work continues, we are already seeing the impact of the new structure. We have been able to deploy employer-specific catalogs, protect content from logged out users, and truly tie the experience of our various applications together.
Finally, beyond the external value that this work has generated, it has also provided a fantastic learning experience for the team. From handling the various aspects of subdomains, to the exploration of Cloudflare Workers — a new tool for our team, to the challenge of orchestrating code and tooling in various places, we learned a great deal as we laid the groundwork for a new and improved user experience.
If you are interested in learning with us and working on a platform that will help millions of working adults go back to school, check out our careers page.
