How to measure WFH employee engagement, and why you should
In my previous blog post, I shared challenges that you might face when getting ready for a company-wide work from home (WFH) shift, and how we at Fiverr IT and Security tackled these challenges by mainly taking into consideration the behavioural change from informal data distribution to formal data distribution (Link).
In this article, I would like to share how the transformation from an informal data distribution to a formal one reflects on IT systems, how you can measure WFH engagement* through IT systems, what you can learn from these measurements, and how IT systems’ statistics can impact management decisions.
You will be surprised by what we have learned…
*engagement and employee engagement have different interpretations by different professionals. My interpretation of employee engagement refers to the reflection of one usage of IT systems
Why is it important to measure engagement?
With the WFH reality forced upon us due to COVID-19, to the evolution of WFH as the new normal, the need to measure employee WFH vs. WFO engagement and productivity became a necessity.
We started the journey of measuring systems’ usage, about a week before the decision to move all employees (globally) to work from home. We conducted a drill, and asked all employees to connect to their systems from home and work for an hour or so — just as if they were in the office.
We needed to know that the systems would be able to bear the load, and that the employees had all the tools they would need to connect and work regularly and seamlessly while at home. Obviously, the fact that we are a “Born in the cloud” IT Systems-based company made the transition easier.
During the drill we gathered data from various systems:
- Unique VPN connection to Dev environment
- Unique Zero Trust access to production and back office applications
- IT Tickets for related connectivity issues (either VPN or Internet)
Post drill we also looked at some additional stats from other systems that we thought were essential to ensuring employees had a painless transition. The logic at that time was to measure the pressure all employees would put on the systems at the same time of the drill
- Zoom
- Okta unique MFA logins
Consider looking for the systems that are used daily, but require the extra effort of opening a laptop, email and messaging systems that are often used on the phone. On the other hand, VPN, ZOOM and Zero Trust access to other production and back office systems, can indicate a more involved engagement.
We decided to focus on statistics from:
- Number of unique VPN connections to dev environment
- Number of unique Zero Trust accesses to production and back office applications
- Number of unique Okta MFA logins
- Number of Zoom meetings per day
We also looked for statistics in other systems, such as IT Helpdesk support tickets, Slack messages and emails, to see the overall changes in data distribution.
From a company perspective, it is easier to make decisions on WFH as a concept if…
- you can reflect how it impacts employee engagement
- you can provide a clear picture over time
- you can correlate between engagement and productivity
This is the type of information that can help a company form a position based on hard data about employees working from home.
What tools should you look into for measuring engagement?
The IT helpdesk ticketing system is always a good place to start — if employees are working, then they have issues — if they care, they will want to resolve them.
We expected to see an increase in IT tickets due to the lack of “informal communication” and necessity to open a “formal” ticket.
Indeed, as expected we can see an increase of 50% to 70% in new tickets per working day opened during March and growing through April and May.
Number of IT tickets per month and avg. tickets per working day
Consider identifying the systems, whose ability to indicate a trend of usage and engagement you are confident in . While an IT Ticketing system can be one, another is the messaging application your company uses. Meaning, the amount of text sent between employees is a strong indicator of “formal communication” used to distribute data.
While it is reasonable to expect more text messaging while working from home, due to the transformation to “formal communication”, if you take the messages sent with correlation to the number of posting members (out of total licensed), it is easy to see high engagement.
With an ~300% increase in Slack messages sent after March 15 (WFH), compared to pre-March 15 (WFO), and no change in the number of posting members, the high engagement is clear.
This trend continued throughout April and May, this is especially relevant considering that the number of active senders did not change.
For comparison, approx 1.2 m messages have been sent each month since March 15, as opposed to ~400k sent in February
Total number of messages sent in Slack before WFH and after
Slack licensed members (in green) vs. posting members (in blue) — nearly 100% of members are posting during weekdays (indicating high engagement
After identifying the systems used for text data distribution, like email, messaging, tickets, etc. You should also take a look at the video conferencing system that you use, in order to compare trends before and after moving to WFH.
Video communication usually requires the use of a laptop for screen sharing and easy video sharing, assuming that sitting for hours in video conference meetings over the phone is not very comfortable.
The increase in Zoom meetings while WFH vs. WFO, is a clear indication of employee engagement, with an avg. of 470 meetings a day through May.
Total number of Zoom meetings per month (red, avg meetings per working day)
Taking a wider angle view over the period of 11 weeks of WFH observed, can provide a clear picture of the company’s employee engagement stability over time. We found a stable pattern of work days, weekends and holidays.
It’s a clear indication that the employees’ engagement was not a periodic transformation, but rather a long and stable adaptation to the new reality.
Engagement graph showing 11 weeks of WFH through 4 systems’ usage
Non-scientific Assumptions:
Following the above, we looked into measuring employee engagement during WFH, and compared it with WFO — but does engagement necessarily mean productivity?
How do you measure productivity? What are the indicators you should choose that can reflect employee productivity during a certain time period?
I gave this one a lot of thought, and discussed it with others, and finally came up with one significant indicator. I’m positive you can find others, and argue about the one that I have chosen.
Feature Releases
A major indicator of a product company, such as Fiverr, is the amount of feature releases made in a specific period of time. Releasing a new feature involves cross-department efforts and can provide reasonable insight into company productivity.
Looking into the engagement increase in the usage of IT systems and the amount of feature releases in the period of WFH, can indicate whether there is a correlation between the measurements of engagement and productivity.
Number of feature releases from Sept 2019 to May 2020
So….what’s next?
Following COVID-19 WFH and the growing global movement toward working from home, be it a complete transition or a hybrid solution of WFH/WFO, it is now obvious that work will never be exactly the same as it was. So, it is important to
- develop the tools that will allow you to monitor engagement
- develop the tools that will allow you to measure productivity
These tools will guide your company in making the right decisions in a world that transforms faster than anticipated, and will allow you to adapt to the future of work and WFH.
