Improving User Login Workflow: The Institution Finder Modal Experiment
With Armaan Dandavati and Evan Shoup
Engineers at ITHAKA are always seeking to improve the user experience of the JSTOR website that allows users to access journals, artwork, and other higher education materials through their local university. One issue that comes up in using JSTOR is the problem of user workflow being interrupted by a login request. Armaan Dandavati (Senior Software Engineer) and Evan Shoup (UI Engineer) are on an engineering team at ITHAKA that ran an experiment called the Institution Finder Modal Experiment, where they improved the functionality of the authentication page when a user is asked to log in through their college or university library. This is how they improved the login function, and what they learned along the way.
Armaan (software engineer):
“In December of 2022, we started talking about ways we could take a user from the turnaway page [on JSTOR] to an authenticated page while keeping them in their flow. Right now we rely on a bunch of redirects. A user will click a button, go to a page with an institution finder, type in their university or college and go to their site to login, and go through redirects to get back to their content.
“User feedback said that takes people out of the mindset of their workflow. To get people closer to their content we suggested popping out the finder in a modal.”
First, the team stuck with the control and pivoted to making it faster for affiliated users to get to their school’s access login. Then they made plans to move the institution finder that is currently a separate page into a modal to keep the user in the experience.
Users off campus could take a long time to log in, and they were leaving the site. The team wanted more users to find their school and log in.
“We worked with a designer who provided us with wire frames. We sat down and paired on this, and it took about a day to do the discovery work. We made a prototype in a separate sandbox environment. We brought it to the team to talk about what we thought the shortcomings were. The team was interested, so we wanted to perform an experiment to see if it would perform better than our current approach. We thought it would perform better, because it takes less time. We used an internal tool to set up our experiment.”
Setting Up the Experiment
Armaan:
“We used an internal tool called the Experiments Service. It’s a really cool tool because it allows you to create an experiment with a control and variants. The control is the current experience. You can set parameters like 50% of people see the variant, and you can actually run it through this tool so you can start it and end the experiment through this tool. Our environment allows us to build two or more versions of the same app and one contains the variant, and then we turn over the hard work to this tool to handle the traffic. The developers can focus on the experience and don’t have to worry about how the [variants are] delivered.
“We were able to whip up this prototype in a technology agnostic way because of this tool. It was fun to build something in a language we were comfortable with and build this the way we wanted.”
Tracking Results of the Experiment
The Experiments Service comes with a tagalong dashboard where notes from the experiment go. The team says the metrics show that users are finding their school more often. The increased usage of pdf downloads of content and secondary button clicks indicate more usage of the login feature that takes users into the JSTOR site to access and download content. They are expecting more users to engage with the Institution Finder Modal now to help them find their institution to log in.
Evan Shoup (UI engineer):
“We had specific data tracking needs where we needed to customize our dashboard. It shows data on the experiment and how it was run. We wanted to look at [the data] to see if the experiment was successful or not.
“We wanted to see: of those people how many clicked on an institution finder result that takes them to their content result? We found that the variant had a much more positive result than the control.
“Based on what we calculated about the test population, 52% of those people converted within a standard margin of error.”
Reflection on The Results
The Institution Finder Modal resulted in more users making it to their institution finder proxy. This is currently being converted into a micro-frontend, which is in line with the JSTOR architecture. The organization recently adopted micro-frontend architecture. The team says this experiment was an opportunity to leverage that new technology and decrease technical debt. The engineers found they enjoyed making improvements that benefited both the website users and the organization.
Armaan:
“I’m most proud of how quickly we were able to whip up a prototype. We were able to bounce ideas off each other and bring it to the team next-day. We were good about not wasting our time. We got something put together to show the team, and then we decided quickly it was viable to put more effort into it.”
Evan:
“I particularly like the fact that it was a user improvement as well as a technical improvement for us. It’s doubly satisfying when you can bring value to users as well as developers on your team. This is going to bring us value in the long run. The interface is easy to work with. Benefits all around.”
What keeps engineers like this working at ITHAKA?
Armaan:
“For me the highlights are the super collaborative experimental environment, so we’re not tied to a particular legacy technology. We can get ideas and iterate from there.”
Evan:
“For me it’s the culture, but especially in UI Engineering and as UI engineers in general. We feel empowered. Delivering on our mission of improving access to knowledge and education can take a lot of forms; we start from the team first. We’re not bound to a specific technical requirement or technology, and it feels good to be part of an organization that cares about you. I think it makes you work better.”
Interested in learning more about working at ITHAKA? Contact recruiting to learn more about ITHAKA engineering jobs.
