How Optimizely used Cypress.io to automate testing within its AppExchange App in Salesforce
This past quarter, Optimizely launched on the Salesforce Appexchange! This was super exciting for the whole team and went smoothly thanks to the e2e testing we had implemented as a Partnerships and QA team. If you’re looking to launch on the AppExchange, it has its own set of challenges, so I wanted to share the Cypress tests I built to do comprehensive testing so you can have a smooth launch on the AppExchange as well! If you have ever been blocked by redirect issues with Cypress, this post should help you out with ways to work around them! The solution may be different but the concepts are still the same!
For those that are unaware of what Cypress.io is, it is primarily an end-to-end (e2e) testing framework used by QA engineers or software developers to write and test their code before shipping these changes to their respective customers. Cypress is integrated directly into your application giving a real feel when running your test suite through a browser of your choice: Chrome, Firefox, or Electron.
In the past, we have primarily used Cypress at Optimizely as a means for regression testing to ensure changes to the system don’t cause previous issues or bugs to reappear. However, we have started to use it to test new features as well due to its ease of use and reliability. We know that if we can test the visual aspects and actions of what our customers will see, we can confidently ship code and features that will deliver a greater experience to them.
In my work as a Software Engineer in Test, I was tasked to pair with our Partnerships team in delivering an e2e test suite on our new product, the Operations Experimentation for Salesforce App (OpX). The new OpX integration makes it easy for customers to run experiments directly in Salesforce Lightning to achieve high-value objectives like quantify the impact of new Salesforce features and test different Salesforce layouts, workflows, and back-end processes.
This was different and exciting for my role because this was the first time I was asked to build a test suite on a product we don’t control (Salesforce). In addition to this, it’s a product being built on a completely different tech stack than our own and by the independent software vendor, CodeScience. In addition to this, I had zero experience working with the Salesforce UI and its authorization flows. We expect all products that have the Optimizely name on them to maintain a high level of quality and OpX is no different. This is where Cypress comes into play.
Cypress makes it very easy to interact with live web pages and getting your first test up and running can take less than 10 minutes. Cypress drove a lot of value by putting a stressor on the product we were delivering. Not only were we able to test the Optimizely Integration itself, but we were able to further validate the Optimizely bucketing mechanism within the integration by running over 10,000 bucketing events through the platform without any veering from expected behavior.
The hardest part of this whole process was the sign inflow. Cypress has a tough time handling cross-domain redirects, which is how Salesforce built it’s login flow. To get around this we had to programmatically log into the Salesforce development environment and store the session data onto the Cypress electron browser. This is done by performing a request (cy.request) to the login endpoint with an access token in hand. We will now go over some steps on how you can get unblocked by the redirect issues.
To do this, you will need to download the SFDX command-line interface (CLI) and connect it to your environment. I used this getting started guide to go through the process.
Once you have installed the CLI to your machine, you can use Cypress to start testing within your Salesforce environment. Here are a couple of ways to do this:
- Run an execute command on the Cypress object calling the SFDX CLI instance for your environment. This will warrant you an access token allowing you to directly access the page being the login flow.
Props to @samuel012 on GH for this easy to use solution. https://github.com/cypress-io/cypress/issues/2367#issuecomment-584058927
I had to tweak a couple of the lines, however, this was the general idea we used to approach the testing.
2. If you are getting a malformed request from the above, you can hard-code the access token from the SFDX and create a login function to handle the request (this example uses some environment variables for the values). Within my cypress.json file in the root directory, I added two more properties for the Salesforce accessToken and frontdoorPath.
“salesforceAccessToken”: “<ACCESS TOKEN FROM CLI HERE>”,
“salesforceFrontdoorPath”: “<SALESFORCE DOMAIN HERE>/secur/frontdoor.jsp?sid=”I created a separate login file to add my login function. This is then requested within the index.js file
Cypress.Commands.add("login", () => {cy.request(Cypress.env("salesforceFrontdoorPath") + Cypress.env("salesforceAccessToken"))});
The one negative of this approach is once the access token expires (Based on your organization’s security settings), you will need to pull a new token from the CLI and paste it into your environment variable. This is not very scalable and potentially makes running in CI/CD impossible.
Once you make a successful request to the authentication server for Salesforce (2xx status code), you can bypass the login screen for salesforce. The world is now your oyster.
I created a few helper functions to work with the common table elements found within the opX UI. If you would like to also automate that your connection is properly set-up, check them out.
The Github Gist for all functions can be found here
Helper function for Optimizely Entity Creation Modal pops up (Project, Experiment, Event)
Cypress.Commands.add('fillOutForm', (selector, index, text) => { cy.get('.isModal').find(selector).then((response) => { cy.get(response[index]).type(text) })})
Helper function for clicking the save button on the entity creation form.
Cypress.Commands.add('clickSave',() => { cy.get('.isModal').find('button').then((response)=>{ cy.get(response[3]).click() })})
Helper function for toggling the metric input box on the event creation form.
Cypress.Commands.add('toggleMetricInput', () => { cy.get('.isModal').find('input').then((response) => { cy.get(response[0]).check(); })})
Helper function for filling out dropdown menus.
Cypress.Commands.add('completeDropdown', (index, text) => { cy.get('.isModal').find('a').then((response) => { cy.get(response[index]).click().then(() => { cy.contains(text).click(); }) }})
Most / if not all of the elements are dynamically generated with a random or class assigned to each element every time you load the page. Since these values change so frequently, I chose to use basic CSS selectors to grab the elements making it easier to test on subsequent loads.
Cypress has validated the great work done by the engineering team and gives us the confidence that our customers who purchase OpX will see a large amount of value in this new capability with Optimizely.
Overall, getting a working testing suite up and running with Cypress was pretty straightforward. I am looking forward to iterating on the suite that I have currently built out and integrating it into the larger tools we are building as a QA Engineering team here at Optimizely.
For more information on how you can run experiments within your Salesforce environment, see here!
To see a demo of Optimizely + Salesforce check out this video below.
Optimizely Operations Experimentation for Salesforce Resources — Optimizely Knowledge Base
