Envoy block list redesign
Many companies purchase visitor management systems like Envoy Visitors to enhance workplace security. They screen visitors to keep unwanted individuals out of their work places to guarantee employee safety, site security, and liability reduction.
In late 2018, I redesigned Envoy’s block list feature with a focus on improving matching, reducing false positives, and empowering workplace security teams to quickly decide if a person should be approved to come onsite. I was transitioned off of this project towards the end due to team re-organization, so my coworker Pedro helped address error states and edge cases that I did not consider, and also made some fantastic refinements to the visitor log UI.
The release of these improvements visibly reduced the false positive rate, and appeared to drive overall adoption of the block list feature. Anecdotally, the redesigned feature was regularly highlighted as a factor that won Envoy new deals over competitors, and reduced churn risk of several existing accounts.
Here’s how we got there:
Background
Customers were excited about the existence of a block list feature, but upon implementation, realized the existing feature was insufficient for preventing unwanted individuals from their sites. Using Envoy Visitors meant they couldn’t meet their security protocols or had to maintain other tools, creating duplicate work for security teams and weakening the appeal of Visitors as a tool. Larger, security-focused companies were unwilling to upgrade or expand Envoy Visitors usage in part because the functionality was not robust enough reasonably address their needs. The lack of functionality was repeatedly coming up in lost deals.
Existing designs
Admins with settings access could add, edit, and remove keywords and reasons for blocking one-by-one to a list shared across a company via the web dashboard. Any keyword added at one location applied to all locations. Admins could also designate individuals to be notified when a match is made.
When the block list was enabled, every visitor’s sign-in information would be matched against the list of filters the admin configured. If a match was made, the designated contact or contacts get emailed a link to the dashboard. They must then go to the Envoy dashboard and approve or deny the entry. Approved matches were differentiated as such, with a record of the admin that approved the sign-in.
The matched visitor is shown an alternative sign in screen telling them to wait. No host notifications were sent out until a match is approved by an admin.
Research & information gathering
“How might we improve the block list?”
Though we knew we wanted to enhance block list functionality and improve visitor screening for customers, the problem space was broad and multi-dimensional, which introduced an even broader set of potential solutions. I conducted research to understand the current functionality, identify and priority-rank customer usage motivators and pain points, as well as competitive weaknesses, and brought recommendations to my PM. These recommendations shaped the eventual product spec.
Feature audit
To start off, I did a full audit of the current feature. This involved mapping out the end-to-end workflows of setting up block list filters, choosing people to notify, testing what happens when a match is made across different admin and user roles for pre-registration and sign-in across the admin dashboard and iPad app.
Primary research
Next, I put together visitor-screening focused interview script and ran ~10 interviews with security and facilities stakeholders from customer companies to understand their use cases, company processes around visitor screening, and their pain points.
The major customer needs and issues were:
- Limited functionality and flexibility (around checking against 3rd party lists, list specificity, notifications, privacy and visibility, etc)
- Lack of integration into customers’ existing security screening tools and processes
- High false positives due to gaps in our matching
- It was hard for admins to tell if the matched person was indeed supposed to be flagged
Secondary research
I conducted a competitive analysis of the existing block list feature against that of 7 competitors. Through this process, I also paid attention to how competitors’ block-list-equivalent functionality worked. My analysis concluded that our block list functionality was limited, not competitive, and essentially overlapped with insights from user interviews.
At the same time, I reviewed our product feedback inbox for relevant feedback and feature requests, and did intakes with our account executives and customer success managers to better understand the specific issues and pains their accounts were having with the block list and visitor screening in general. I grouped feedback into themes.
Data insights
Data investigation showed that of the companies that used block list actively, there was around a 77% false positive rate, confirming unreliable matching. The majority of matches were approved or denied within 60 seconds, and the response time was long-tailed.
A lot of companies turned it on and quickly turned it back off, indicating they may have had interest but didn’t see enough value for active use.
Problem definition
Equipped with insights from research and better understanding current shortfalls that prevented customers from confidently keeping unwanted people off-site, we honed in on problems to focus on, in ranked order.
1. Inability to use photos/visual information to verify the match. Words aren’t sufficient to determine if a person should be blocked from coming onsite — users need visual information to more confidently make a decision to approve or deny the person. With the current product, users feel stuck when the name is a match — they have to use a judgment call to decide whether to approve/deny the person. Several customers asked for the ability to upload a photo for each person on the block list.
2. Need more attributes about the blocked person to make a decision. Users don’t have enough information and attributes about the blocked person to confidently make a decision about whether to approve/deny the person. Users asked for a way to add descriptive information about the blocked person, such as name, aliases, address, phone number, email address, date of birth, company, physical descriptors, etc.
3. High rates of false positive matches. Around 77% of block list flags are approved for entry, indicating false positives. Building security teams are alerted about each block list match, and are required to investigate each incident. A large number of false positive matches causes additional, unnecessary work for these teams. Some customers want to set rules so that keywords only apply to specific sign-in fields. This way, users can more accurately configure what rules need to be met in order for the block list to flag that a match has occurred.
Explorations
I had a lot of ideas around how to configure the setting and how to surface matches, which I explored to varying depth. I went through numerous rounds of feedback and rapid iteration with my cross functional team, design team, and other stakeholders.
Creating block list rules
I had originally identified two use cases for creating blocking rules — blocking people and blocking characteristics (like company or answer to a field) but eventually moved towards the mental modal of blocking people, while technically still allowing customers to set general keywords to trigger matches within the design that eventually shipped. We also came up with an unintrusive migration plan for the block list filters customers’ had already set. Something else I explored was creating and editing rules inline, however, given the sensitive and critical nature of visitor screening for companies, I decided to add an explicit edit action.
Surfacing matches
A substantial portion of this feature boiled down to aiding decision-making. How could we surface the most relevant information to admins making approval and denial decisions? Some ideas here included splitting up the visitor log to better highlight visitor entries that required attention. Given some data insights that 90% of approval and denial decisions were made via the visitor log, I also played with other ways to surface more information without adding too much cognitive clutter, such as a card with more info that appeared upon hover, and a flyaway component to surface match information.
Feedback and iteration
User testing
I conducted usability sessions with several of our customers. I showed them a prototype of the feature and asked them to talk through how they would go about completing tasks like adding a blocked person, and taking action if a match was made. A lot of feedback was related to additional functionality that was out of scope and thus was not immediately actionable, but below is some recurring feedback and subsequent design changes:
- Some customers wanted to see more details about a match, but weren’t sure if the matched entry in the visitor log was clickable. To make this more clear, I added a clearer CTA to indicate you could click.
- Some customers expressed the desire for a larger photo reference, so I increased the size of the photo on the visitor details page.
- Some customers were confused about the “keywords” field, and uncertain if they could add multiple items for some fields, so I added help text to the empty fields when a new blocked person was created to better indicate that multiple were possible.
- Based on feedback on the value on the “reason for blocking” in making decisions, I adjusted the design to make it more prominent on the log, and adjusted the grouping of “reason for blocking” on the visitor details page so it would be close to the other information the admin would be consulting to make a decision.
Design critique
I asked my team to focus feedback on getting the right balance of information to help admins make approval decisions quickly, but not get overwhelmed by information. I made several changes focused around information hierarchy, and clarity. I played with how to distinguish matched entries in the visitor log, how much information to surface in the log, and explored some new UI elements that would provide more information. Getting feedback from my team was especially helpful in honing in on a way to surface matched fields.
While initially I had designed for the log view to be more of a summary, assuming that the admin would click into the visitor details page if they needed more information, based on data findings that 90% of approval or denial decisions were made on the visitor log page, I added more details.
Shipped solution
This project spanned across the web dashboard, iPad app, and notification emails. If you’re interested, you can learn more about how the block list feature works.
Performance
90 days after launch, the feature exceeded the goal we set for increase in overall usage of the block list feature. We also saw a visible reduction in the false positive rate from 77% to 30%, indicating that the new design was much better at correctly matching blocked individuals. However, many companies using the block list did not update their filters after the migration. Either many customers didn’t care enough to move their filters over, found it laborious to update their filters, or didn’t yet see the value in doing so.
Anecdotally, improvements to the block list feature were specifically highlighted as factors that won Envoy new deals over competitors, and reduced churn risk of several existing accounts.
Next steps
Based on customer feedback in the year since launch, next steps would be to explore ways to make it easier to set up (eg bulk creating blocked entries or supporting uploads) and manage block list filters to alleviate the low adoption of v2 filters by existing block list users. I would have been interested in exploring supporting approvals and denials directly from the admin notifications to reduce the time it takes for an admin to be able to act. Other value-add ideas were supporting location-specific lists, assigning priority levels to blocked persons, and making notifications more flexible and granular.
There’s a lot of room to iterate on this feature to make it extremely powerful in improving workplace security and increase its power to win deals.
Takeaways
Research is an investment in the future
Being intimately familiar with customers’ use cases, the competitive landscape, and all of Envoy’s competitive weaknesses helped me become a source of knowledge for this project. From my research, I was able to bring recommendations on what problems we should address to my team, which greatly informed the product requirements and final shipped feature. It was incredibly empowering to be able to refer to research insights when presenting designs and advocating for certain things, and significantly de-risked the project. Having a clear picture of problems and priorities made the rest of the process much clearer.
Cross-functional communication needs to be designed too
At the time, this was one of the largest, most sprawling projects I had worked on to date. The workflows spanned across platforms and user permissions, and involved a large body of edge cases. Regularly checking in with my PM and eng lead helped keep the team on the same page. In my previous projects, I’d noticed that design intent wasn’t always fully captured while doing design QA, which would lead to avoidable visual bugs. I also received feedback from engineers that our “handoff” tool, Zeplin, wasn’t clear in communicating user flows and that notes would often get buried. Given the complexity and breadth of this project, it was especially crucial to fully communicate design intent and minimize ambiguity. I produced a variety of additional artifacts like flowcharts, screen maps, and clickable prototypes. I also added note boxes directly on my high fidelity screens and recorded a video walk through for the engineers to reference, to positive feedback. Working directly with engineers when they had questions, and also doing rolling design QA as dev work was being completed allowed for close collaboration, which I think helped the implementation of this feature.
Functionally successful features don’t always translate to high adoption
It’s unrealistic to cling onto the old adage “if you build it, they will come.” Despite the noticeable matching improvements and the overall benefits of a more robust block list feature, adoption among customers already using block list was relatively low. Internally, we had hypothesized customers didn’t care enough to move their filters over, found it laborious to update their filters, or didn’t yet see the value in doing so. While we aren’t fully clear on the primary causes of low adoption, given all of the development effort that went into this project, I wish there was a more concerted product marketing initiative and customer enablement strategy, especially given the ample data we had on customers interested in this functionality. Because, at the end of the day, features don’t have real value unless they are being used.
Fun fact : Block list was previously known as “blacklist”. As part of the product work here, we also incorporated user feedback around the problematic language of “blacklist” and renamed the feature.
Thanks for reading! Be sure to visit envoy.design and subscribe to get notified when we publish something new.
