Serverless landing page optimization: implementing rapid experimentation at Opendoor.

Josiah Grace
Apr 6, 2020 · 6 min read

This is the first in a series highlighting Opendoor’s use of Cloudflare Workers. If at any point you want to jump straight to working code, it is available here (and in the implementation section).

Opendoor is coming up on two years of running Cloudflare Workers in production. Cloudflare Workers are a serverless runtime, which allow for a bit of Javascript to intercept, execute, and modify incoming requests at the CDN level (see Ashley Williams’ talk at JSConf EU for an excellent overview). Workers are lightweight enough to run before every request and provide powerful building blocks to perform common operations like proxying, setting headers, and the like. At Opendoor, workers were originally deployed to handle a single use-case, proxying requests from* to our internal Wordpress instance. Our worker allows for bundling of required npm packages, has full test coverage, and is actively iterated on by developers at Opendoor every day. (Bonus! It’s written in Typescript, and Opendoor ❤s Typescript). Over time, their utility, ease of use, and continually expanding capabilities led to an explosion of use-cases; they now run before every request to an Opendoor web property.

Cloudflare Workers power our landing page infrastructure and enabled us to:

  1. Run an A/B test across old and new infrastructure
  2. Run an A/B test across two separate landing page designs
  3. Run multiple A/B tests on different variations of our new design
  4. Simplify developer experience on the new infrastructure

All with a pretty small amount of experimentation/routing code. We hope you all are as excited as we are about this approach and the power of workers.


  1. Rewrite our landing pages with a new infrastructure/framework (optimized for page-speed improvements, in this case, nextjs) with the existing design and roll this out (ensuring conversion was flat or positive).
  2. A/B test the new design vs. the old with the new infrastructure
  3. Run multiple experiments on the winning variation to boost conversion

All of these steps were ordered such that we would first validate our new infrastructure, then our new landing page design, and finally, experiment on variations of the winning design.


  1. Identify the user (or generate an identity if they’re a new user)
  2. Choose the experiment group the user is assigned to (or was previously assigned to), and satisfy their request for that content
  3. Log the user and variation for use in conversion analysis

The two commonalities across all experiments are the experimentation platform (Optimizely) and a distinct user id (held in a first-party cookie on the domain). The three A/B tested domains are infrastructure, design, and variation. Each of these need to be rolled-out, measured, and either A or B is declared the winner. We can not mix and match these three categories or there are too many variables to isolate in analysis. In order to accomplish these goals, we have to decide where the shared infrastructure lives.

Our first experiment domain is the old vs. new infrastructure. These are two separately deployed services, the first is served from our heroku deployed rails monolith, the second is a kubernetes hosted node service running server-side rendered nextjs — these differ by host (e.g. vs. Next, our design test happens on the selected serving infrastructure. Each design variant will be on the same host but won’t share much code. Finally, we’ll test variations on the winning design (headlines, calls to action, and the like). These will be mostly the same code with conditional variations. We have two options here, given these requirements.

Option 1: Experiment logic in application code

If the experiment logic lives in the application code, then an infrastructure test would not work, as we’d have to wait until a backend was fulfilling the request before determining which infrastructure to use to fulfill it. If we’re on new infrastructure, we could assign in application code, but then we couldn’t cache requests to our backend, because we’d have to hit our upstream to determine the experiment group. Developers would also have to fake user information and experiment groups while developing.

Option 2: Experiment logic before hitting application code

If the experiment logic runs before application code, we are able to cache requests to our backend and determine which infrastructure to hit before being in application code. This introduces the requirement that our request URL is the only data dependency for the backend. This requirement also simplifies developer experience, as there’s no need to mock or fake experiment groups while developing. The nextjs backend doesn’t even know who the user is, it renders solely based on the request URL.

Decision: Option 2

Since Cloudflare Workers can run before every request, fetch different backends, read and set user cookies, and load and run the Optimizely Javascript sdk, we used them to implement option 2. When a request comes in, we check a user’s experiment groups then format an upstream request with the worker. The worker changes the host for an infrastructure test, the path for a design test, and query parameters for a variation test. Since the request URL is the only data dependency, this is also all cached at the edge.

Visualization of selected architecture (option 2)


  1. Match the request against the landing page path (GET + $landing_page_path)
  2. Parse the “anonymous_id” cookie (generate a random uuid if it isn’t present)
  3. Check what experiment group the user is in (this is a stable assignment based on user-id)
  4. Fetch the content for the url associated with that group (and return the response to the user, usually cached)
  5. Log the experiment assignment

A full working example of the concepts in this blog post (including sample infrastructure, design, and variation tests) is available as a Cloudflare Workers template using the following commands. This includes all the configuration necessary to integrate Optimizely with a worker (including a working webpack config, bundled libraries, and cookie management):

An simplified example of the worker code for this is as follows:

If you have any questions or feedback, feel free to open up an issue on the linked repository or reach out on twitter (@defjosiah).


Stay Tuned

If you’re interested in this type of work, Opendoor is hiring engineers! Head to our careers page to learn more.

Open House

The Opendoor Engineering and Data Science Blog

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store