Getting Started with Klaviyo as a Developer: Testing and Go-Live
This is the third article in our Getting Started as a Klaviyo Developer four-part series.
So far, we have discussed modeling data and workflows and mapping models to Klaviyo. Now it is time to discuss strategies and considerations for testing and go-live.
Overview
- By using unit tests and end-to-end tests, you will feel confident in your code.
- Klaviyo offers several tools within the application to help scaffold data and create a testing sandbox.
- There are several product features to preview and test content to be confident before sending.
Testing: Building confidence in your code
Before going live, it’s important to have confidence that what you built will work in production. The best way to gain this confidence is by thorough testing of the logical components of your system, as well as end-to-end testing with connected applications. We’ll cover the tools and strategies you can use to build robust testing.
Unit testing
One great way to build confidence in your system is to continually test it as you make progress. At the code level, this includes unit tests.
One of the most famous examples of extensive test coverage is found with SQLite. It contains roughly 7 times the amount of test code than non test code, and achieves 100% modified condition/decision coverage (you can read more about it in this comment from the creator). Even then, one key takeaway from the SQLite project is that “there will always be bugs”. But with good testing, you can feel confident you have eliminated as many bugs as possible.
We highly recommend unit tests as well as end-to-end tests for your systems, which we will discuss below.
End-to-end testing with Klaviyo
Klaviyo offers several tools and strategies to help you get started with testing. As a data-driven platform, creating a sandbox account with mock data can help mimic a back-end integration for testing of APIs and product features. We also offer a front-end library with robust features to connect a front-end website to Klaviyo. This will allow you to get an account set up quickly with both back-end and front-end data so that you can be ready to build.
Tools to set up testing
When building a system that integrates with Klaviyo, it is important to create end-to-end testing environments so you can feel confident before you go live.
When working with disparate systems and applications, it can be challenging to set up proper tests. Here are some tips and tools to accomplish this using Klaviyo:
- Create a sandbox environment — Sandboxes are full-fledged accounts with all of the same features as a paid account. Using a sandbox environment is helpful to avoid adding sample data to a production environment and to avoid any impact to deliverability.
— Review the linked guide for tips on how to effectively use a sandbox account. - Generate sample data — If you are looking to test product features, or GET data from API endpoints, sample data will help you get going faster. You can populate different types of sample data: Klaviyo, eCommerce, and catalog.
Note: The data generation tool can only be used to create sample data one time per data type. Any data not yet generated can be created at a later time by re-accessing the tool. Data types that were already created will be disabled.
- Confirm front-end tracking is enabled on your website — Viewing a website with klaviyo.js tracking enabled and adding the query param: ?utm_email={your_email_address} with your email (and no brackets {} ) will cookie your browser and identify you to Klaviyo. After loading klaviyo.js on your website, you can access the klaviyo object. This is part of our new client-side APIs and provides enhanced functionality over previous client implementations, notably callbacks and promises so you can handle various return types for different operations. For testing and debugging front-end profile / subscription / event API calls, the linked article describes how to ensure a ‘Viewed Product’ event is working, which relates to any front-end event.
For front-end development sites that do not enable https (e.g. localhost), you’ll receive a CORS error from the Klaviyo APIs, to work around this:
- enable https for localhost
- use a proxy (e.g. ngrok)
- temporarily disable CORS in your browser (and remember to turn it on again!)
Teardown testing
A common approach when testing is to build, test, and then teardown afterwards. This way the tests are idempotent and can be re-run without worrying about leftover artifacts. This can be accomplished by deleting profiles from their corresponding profile view after they are created:
The workflow looks like this:
Testing features in Klaviyo
There are several tools and areas of the product that include testing functionality to feel confident before turning things live. We’ll discuss where in both the UI and API to find these.
Testing profiles
When creating or updating a profile in Klaviyo using the APIs, you can use the Profiles endpoint for CRUD functionality for Profiles.
Within the Klaviyo UI, you can use the search bar with a customer identifier (e.g. email address, phone number, etc.) to find the customer, as shown below.
Testing segmentable properties
All data that is stored at a ‘top-level’ in Klaviyo is available for segments and flow filters. Klaviyo’s segmentation engine stores and updates new data in real-time, so as soon as you send data into Klaviyo, you can build a segment using it.
For example, if you create a ‘Placed Order’ event via API when someone completes a purchase on your site, you may want to filter events based on what item(s) someone purchased. To do this, bring data from nested structures to the top of the event object. If there are multiple items in an order, this is typically accomplished by using top-level array data structures and concatenating the items in those arrays.
To confirm that Profile and Event data is stored appropriately, it is good to confirm by building a segment and checking that you see the expected profiles.
Note: If you’d like to remap nested properties as top level properties, you can accomplish this by using Klaviyo flow webhooks functionality.
Testing flows and messages
To confirm that Flows and corresponding message templates are coming through as expected, Klaviyo has preview tools that can be used to check message contents.
- Navigate to Flows, and select a message within the Flow to preview how that email or SMS message would look for different profiles and events for the flow
This is a form of user-acceptance testing that allows you to confirm that the user experience for your system is meeting expectations. Both of these preview tools allow you to see the last 10 events that match the flow trigger and profile properties for profiles you can choose. If you are close to going live, you can set flows to ‘manual’ mode, which will allow you to manually trigger each message for users who qualify for messages, and will not automatically send anything. This allows you to screen the first series of automated messages before sending, and confirm any flow filtering and conditional logic results in the right evaluations of who should receive messages.
If you are very close to going live, and have production-level events firing into a manual flow, you may have too much incoming data for the “last 10 events” preview window. If this is happening, it can be helpful to send test events with a different Event name, so you are not in the same flow as your high-volume production data.
Preview of the SMS:
Preview of the email button:
Preview of the email interface:
Testing campaigns
This is a very straightforward process. You can create a list or segment of test users and send the initial campaign to this group. Afterwards, you can clone the campaign to your production account to send it to the intended recipients.
Testing API requests
When testing synchronous API requests to Klaviyo, the response should provide enough detail to feel confident that the API request was successful. This eliminates the need to chain GET requests to confirm the item was created. Requests can be sent to create/update data in Klaviyo, the response handled, and then you can delete the data as part of a teardown process.
For testing asynchronous requests, it is a little trickier. Depending on the endpoint and the amount of data that is being sent, it will take some time for the request to fully process. We recommend a) testing your request and processing time, b) making a request to check the asynchronous job’s progress on 1 minute intervals, and then c) deleting the resource once it has been processed.
A tip when building these teardown tests is to first make a GET request before each test run, to make sure that the resource was successfully deleted (i.e. a GET returns a 404) after the last run. For synchronous DELETE requests, the response will provide you instant feedback, so the initial GET is unnecessary.
Go-Live
Klaviyo offers several features to get you confidently to the starting line. Our product features that render on your website (e.g. signup forms) and send messages (e.g. campaigns, flows) all include multiple publish statuses, to prevent you from turning things live before they are ready. These statuses are Draft, Manual, and Live. Let’s walk through the various publish statuses for flows, forms, and campaigns.
Flows
When a flow is in Draft mode, it will not trigger users. In manual mode, users that qualify for the flow and corresponding steps will be queued, but the Klaviyo account user (e.g. you) will need to manually send any automations. Live mode will send messages, so make sure you’re ready before updating the mode to Live!
Forms
For sign-up forms, ‘Draft’ mode will allow you to edit the form without displaying it on your website. Updating the form to ‘Live’ will result in it rendering on your site. There are many advanced targeting options for forms, including by segment and to specific pages. One common strategy to double-check rendering on your site when going to ‘Live’ mode is to render a form only on a test page URL and/or with a special query parameter at the end of the URL. To do this, you would update the form to display only for certain URLs, and then add “#” and that string to the URL to trigger the form, such as www.example.com/#testing-form. Then make sure to update it once you have confirmed it is rendering properly before going live.
You can also render a form only for users in a specific segment, and create a segment with only your profile. Make sure to update that segment to your target segment(s) when you’re done testing.
Campaigns
For testing campaigns, the process is very straightforward and there are two options.
- Directly within the email template editor, you can send preview emails.
— This allows you to send the email directly without scheduling and sending a campaign. - Create a campaign and then send it to a list or segment of your own test email addresses.
— From here, clone the campaign when sending it to the real audiences and choose the segments/lists you want to send and exclude from it.
Conclusion
When testing end-to-end with Klaviyo, there are several in-app tools and features that you can use to feel confident that the system you built is working as intended.
- Unit and integration tests within your codebase, and expanding that into end-to-end tests with Klaviyo.
- Teardown tests are a powerful way to create an idempotent comprehensive test suite, after creating Profiles and Events in Klaviyo, you can delete them to teardown and be prepared for the next test run.
- Using message previews in flows / signup forms / campaigns is a powerful way to build user-acceptance testing into your process as well as collaborate with business stakeholders to confirm that requirements are being met.
If you would like to keep up with the latest news/events/happenings please sign up for the Klaviyo Developer Newsletter.
