Small feature, big impact
How we do it at SmartRecruiters
Abstract
It’s a story of a small product enhancement which had big impact on our customers thanks to continuous monitoring adoption and reacting with certain improvement when a need was detected. The story proves that sometimes to be successful it’s not enough to have a great idea. Promotion is crucially important too.
The Beginning
It all started back in December 2017. An idea crossed my mind during one of the product discovery meetings we do at SmartRecruiters. Though it was unrelated to the topic of the conversation we just had, I shared the idea with Natalia, our Director of Product Management who was visiting from San Francisco.
Natalia’s absolute positive reaction was a true kickstart point for project! Soon after Piotrek (a Front-End Engineer) joined our team, the “conspiracy” began…
The Idea
Recruiters needed a way to “favourite” a particular page of our platform so that they can quickly go back to the work they had to stop or as a reminder of things which remain to be done.
The idea was simple: to make it possible for our users to mark any page of the SmartRecruiters platform one has access to as a homepage which is being displayed right after successful login. It sounds like pretty straightforward functionality, especially when every modern browser comes with support for bookmarks. However, as the feature adoption showed in the end it was a bull’s eye!
Product Process in a nutshell
The feature was pretty small, compared to other projects, but we were still guided by our product process. This process usually starts with discovery, then user testing, collaborating on the feature design, coding, shipping and then releasing the feature.
Making it happen
After gathering more positive and constructive feedback from several people we validated the feature’s potential and decided to proceed. We invited Shefali, a Product Designer from SF to join our case. Soon after we had a beautiful design ready and we were ready to code it!
Here is how it looked like in the initial release:
It was the beginning of a January 2018, so we needed to hurry to make it ready for the upcoming Winter Release of the SmartRecruiters platform on the 25th of January.
A week before deadline, the “conspiracy” ended as the feature was publicly announced during a regular pre-release webinar organised for our customers. It was presented as a bonus, secret feature, a product enhancement suggested and implemented by the developers. It received an enthusiastic welcome, and that gave us wings for further work.
Thanks to the team’s great contributions (especially Piotrek), we managed to deploy the feature on production and on time despite being involved in other more significant development activities for the same release.
At this point, it was live, and we were super happy with it! But, that’s not the end of the story.
Watching it grow
We have it in our blood that once a feature is release on production we begin to monitor its adoption. This case was not different.
By the first day, around 80 users set a custom landing page. A few days later, the number had jumped to 176. Unfortunately the feature adoption wasn’t increasing as we wanted it to. There was an initial, post-release boom but then in a one week time we saw our feature “dying” and decided to act!
It took us some time to come up with an idea and actually implement it. In the meantime more and more users were using the feature but still we were unsatisfied. After a month, we had 400 users, but we wanted more…
Finally on the 4th of March we released a small (but as it turned out significant) improvement. Whenever a user stayed on a page for longer than a minute, a tooltip popped-up over the bookmark icon. It said:
Looks like this could be your new homepage. Click this icon to change it.
We didn’t want the tooltip to be intrusive, so we decided on a set of rules for when it should be shown to users. The tooltip would get shown for a 2-week period, only to those users who hadn’t used our feature yet and no more than once a day per user. We also set a 1 minute inactivity delay to detect when user remained on a particular page.
We deployed it and watched, hoping the adoption would change. What we experienced exceeded our expectations.
During the two weeks when the tooltip appeared, over 300 new users started using the bookmark feature. This growth double the users from the entire previous month. This was definitely a result of an increased feature awareness.
We received further validation that the automatic pop-up had a big influence on adoption when we noticed that user adoption slowed down noticeably when we remove it.
The Boom
After the tooltip experiment we left the feature on its own checking the adoption from time to time. 3 months after the release, on the 24th of April the number of users using the feature reached 980. We were one step from crossing the magical 1k border!
Around the same time, we introduced a new design for our platform’s top navigation bar. This was a first step in our ongoing initiative to introduce a new platform-wide UI redesign, known as “Blue Steel”.
Thanks to Piotrek’s own initiative the bookmark tooltip was restored as a part of that work and it turned out to be an excellent decision. Right after the release adoption rate was higher then ever!
Since the tooltip mechanism and conditions for displaying it didn’t change, it must have been the buzz around “Blue Steel” that caused the effect. It may also be the fact that thanks to the completely different palette of “Blue Steel” and its better contrast, the tooltip was more eye-catching.
Whatever the reason, we’ve reached our goal of having 1k of active users with a custom landing page. Woohoo!
The Elephant Effect
That wasn’t the last word of our tiny feature, though…
A couple weeks after, on Monday the 14th of May 2018, one of our biggest customers went live and additional thousands of new users started to use our product. Since the feature was already popular among that particular customer users and general feature awareness was pretty high, it resulted in a huge increase in our adoption rate.
Today, we have over 8725 users using the feature. That’s more than 8x our original goal!
Summary
The Custom Landing Page success proves that without appropriate promotion even a great idea may have no impact on your product.
Its adoption rate changed dynamically over time. There was a moment when we considered it dead and decided to act. In-product tooltip and suggestion turned out to be the right way of improve our feature awareness. This had a direct impact on the number of (hopefully) satisfied users who made of the enhancement.
So if you’re building a feature you’re proud of, no matter how small it is, make sure to work on its discoverability and awareness as without those two factors all the effort you took may be in vain. Make it your habit or even make it a part of your product process to measure and monitor feature adoption so you can react quickly whenever something wrong happens to it!
Our small feature is also an excellent example of a bottom-up initiative which is essential for a success of every company. At SmartRecruiters, we are aware of how important it is to create an environment which supports developers so that they can make their ideas come true.
Currently the process is more structured as every team gets 2 weeks of Delight Sprints every quarter to work on such platform improvements. It’s up to them and their Product Manager to decide on the thing they would like to implement.
The Bet
A fun fact about the project is that at the early stage Natalia made a bet with Jerome (CEO of SmartRecruiters) regarding the future feature adoption level. Eventually she won and I hope she already collected her 20$ from him ;)
Despite losing some money I do believe Jerome was satisfied with the outcome as in general… everybody won. Especially our customers.
