Numbers: One Week into the Launch of our Redesigned Intranet and ESN

What they are saying and what they aren’t saying…

Numbers by themselves rarely tell a complete story. And sometimes the things we can measure aren’t necessarily the things we should measure.

We tend to overvalue the things we can measure and undervalue the things we cannot. — John Hayes, CMO American Express

Laurence Lock Lee wrote about the pitfalls of using so-called vanity metrics (likes, comments, shares, etc.) to gauge success in a CMS Wire article back in June this year, which covers the topic very well and convincingly.

However, after only a week from launching our redesigned intranet and ESN platform, IBM Connections, out-of-the-box vanity metrics is all I have to go on. I’d rather look at an incomplete gauge rather than no gauge at all. So, here are a few of the metrics that point to how our employees are taking to the redesign and new collaboration platform.

Intranet

The intranet isn’t new to our employees, but it has been quiet a while since the last major user interface changes. About six years since we added the blog, but really very few other design changes were implemented even over the years prior to that. There are many resources that our operational employees access on a daily basis on the intranet, so it isn’t really an optional site for them to visit nor is it one that they have to access only on a set schedule, such as for annual benefits enrollment, etc.

Last week we saw about a 16% bump in visits over a seven day period. Launch day traffic accounted for a large portion of the increase, with about an extra 40–50,000 visits.

For the remainder of the week, visits were up slightly but nothing significant.

When we looked at our top ten page views, we saw many of the usual suspects. However, we had two new additions and one glaring omission.

One of the new features we launched was a toolbar that employees can customize by adding a few specific applications and/or pages to. The page to add and edit selections landed at number six on the top ten list. This was very encouraging!

Rounding out the list was the blog post announcing the redesign and launch of Connections. Very rarely, if ever, does one of our blog articles get enough views to make the top ten pages visited. This post saw nearly a 100% increase in views compared to the yearly average.

One curious item that was missing was one of our pages that is typically in our top three — the page that launches web-based email. We put an email button in our toolbar, so it looks like a) people saw it and used it, and b) we are going to have to come up with a way to measure that if we want to keep tracking it.

These data points tell me a couple of things (or at least, I’m going to make some wild ass guesses about what I think it tells me):

• Our efforts to promote the launch and make employees aware paid off. They showed up in mass on launch day.
• The redesign wasn’t violently rejected. I never really thought that our employees would truly “reject” the redesign, but the apprehension and anxiousness was present none the less.
• Employees were at least curious about the new toolbar feature. Additionally, about a quarter of the tickets we received last week were requests for additions to the selections available in the toolbar. Sounds like the toolbar is a hit and we need to keep making it better for our users.
• People were interested in the redesign as shown by the large amount of views to the announcement post. Based on the sentiment of the comments we received for that post, the majority were neutral (which is typical), but we ended up with more positives than negatives over the course of the week. Negative comments were more prevalent initially as I think should be expected.
• People get used to website changes pretty quickly. This may be a conclusion that is a little premature in making, but I’ll do it anyway. Traffic stabilized. Negative comments subsided (some people even came back and recanted!). And, the normal resources that our users access were the same things accessed last week as the weeks prior.

ESN

IBM Connections — any enterprise-wide internal social — was a wholly new application for our employees. Contrasted to our intranet, no one started day one out with a work related necessity to use Connections. The closest requirement for accessing Connections on day one was that the profile photo used throughout the intranet and synchronized down to email and instant messaging was the profile photo housed in Connections. We loaded badge photos for the initial picture and allow employees to upload a new photo if they choose. We only ask that it remain an actual photo of the employee so their coworkers can easily identify them.

Over the course of the week, Monday through Sunday, we had a total number of visits in excess of 56,000. For unique visits, we had around 10% of our workforce hit the site daily. Weekend traffic was expectedly low; however, activity (updates) occurred on a fairly constant basis. We had over 21,000 updates in the first week.

We hit a couple of goals that we established much quicker than we had set and have one left to attain.

Encouraging employees to update their profile with a new photo, adding in background and career history information, and connecting with other employees is an important component to our adoption strategy. We wanted to get 5,000 profile updates. We hit that mark on day three and ended the week with over 9,000 updates.

A second goal we established was to get 10,000 visits to the Connections “homepage” or activity stream. This gives us an indication of repeat users since someone’s first visit could be from a link that a coworker shared with them or from one of the automatic notifications Connections sends, such as an @mention or being added or invited to a community. We surpassed that goal by day five and ended the week with nearly 12,000 “homepage” visits.

The last goal we set and are still working to is to get employees to download and setup the native mobile applications for Connections. We set the bar fairly low on this one at 500 devices activated and are currently at a little under 400. I think it will take a couple of weeks to hit this one, but we have a couple of “expos” coming up that I think will help promote mobile adoption.

More assumptions and conclusions:

• People were curious about this new collaboration platform they had been hearing about! We had the most visits on launch day, again, as should be expected.
• For employees that are using the platform, they are using it multiple times a day. Our overall visits metrics are consistently much higher than our unique visitors metrics.
• Mobile adoption is slow. This could be for a number of reasons, but I’ll wager some guesses. Setup requires putting in a URL that is longer than can be easily remembered or typed out, which is a small but noticeable barrier in the process. It isn’t just typing in a username and password (but they have to do that too). Most people don’t readily install apps on their devices anymore. They need a reason to do so, and the reason for installing this new platform hasn’t been established for them yet.
• If we can keep up the current pace of profile updates and homepage visits, we are going to have a healthy user base by the end of the year. I think this one is wishful thinking on my part. I should expect to see a dip in the coming weeks for new users accessing the tool for the first time and kicking the tires. We are getting the early adopters now but will have to put in some extra effort to get real traction with the early majority.

Conclusions

I’ve made enough uninformed conclusions off of incomplete data already. I’ll save any more for a later date!