Insights from An IBM Enablement Study
In May of 2021, IBM hosted its first ever internal design conference: SPARK. This three-day virtual conference was organized by IBM designers to inspire and learn from each other, and I had the honor of presenting a recorded talk on designer and developer collaboration alongside Bradley Knaysi.
The talk, entitled “Enabling meaningful end-user experiences through designer and developer collaboration,” shared our collaboration strategies that both worked and didn’t work on a recent usability project of the REXX mainframe language in IBM Z Open Editor, a new generation mainframe development editor.
Throughout this usability study, our primary focus was to figure out how we could make an Outline View useful in IBM Z Open Editor. Through our successes and failures, we came away with three valuable takeaways that contributed to an overall success.
Understanding the study
Before diving into the details of our study, let’s begin with REXX.
What is REXX?
REXX (Restructured Extended Executor) is a mainframe programming language developed at IBM. It is a structured, high-level programming language designed for ease of learning and reading.
REXX is used as a scripting and macro language, and is often used for processing data and text and generating reports.
With REXX in mind, this study had two primary needs: a customer need and a business need.
Customer need
Mainframe developers do not currently have a modern mainframe editor choice for REXX programs. By modern mainframe developer experience, developers have to write REXX mainframe code without features such as syntax highlighting, checking for syntax errors, code completion etc. This decreases the productivity of early-hire mainframe developers who are used to working in modern development editors like VS Code and IntelliJ.
Business need
Seeing the market opportunity to help the developers in IBM Z Client organizations, IBM Z started a Bring Your Own IDE (BYOI) option for application developers to develop mainframe application components in a development environment they are familiar with.
BYOI means that mainframe developers in client organizations have access to different mainframe development editors. This BYOI initiative helps the client mainframe teams add more developers to their teams quickly and provides a choice of different mainframe development editors. It also encourages developers in other parts of the organization to write code for mainframe applications in the development editor of their choice.
Although both the customer need and business need gave us a roadmap, we faced constraints before beginning our study: we had a limited amount of time before the beta version of the product, and Brad had limited exposure to REXX.
Comparison of two mainframe editors
The two photos above show different editing experiences for REXX. The photo on the left is the most popular legacy REXX editor in ISPF, while photo on the right shows the most modern REXX editing experience in Visual Studio Code. The difference between these two editors is quite stark. Despite the fact that ISPF has existed for over 30 years and currently lacks many common editor features, a majority of the mainframe industry still relies on ISPF for their daily development.
In the example program shown, the Outline View simply displays the internal routine definitions to the user, providing them a high-level outline of the program, as well a mechanism for quick navigation. In ISPF, similar navigation would require executing commands to find potential matches, and then tabbing to find the right occurrence.
Writing user tasks
One thing that really helped our collaboration techniques was when I guided Brad through design exercises. To help him develop the correct frame of reference, Brad would draft tasks, and then we would discuss their strengths and weaknesses. This also helped Brad understand the philosophy behind user tasks — what makes them good and bad — and the common pitfalls to avoid.
Slack conversations
I created group Slack conversations with the participants to help them get the IBM Z Open Editor extension installed and configured in VS Code prior to the usability session. This was a great way for Brad to engage with the users, and taught him the importance of writing unambiguous directions. It was interesting to see where people got stuck, and then iterate and morph the instructions over time to be clearer.
Pilot and main study
In our pilot study, through my recommendation, Brad had to resist the urge to help the user when they struggled. Instead, he observed their behavior to identify the pain points hiding between clicking and scrolling.
In the main study, I gave Brad the flexibility to go off script and ask technical questions when something seemed important to dive deeper into. There’s a delicate balance between breaking the flow of a session and enriching it, but these in-the-moment type questions can bring common understanding to both interviewer and interviewee.
As a developer, it’s important to be extroverted during the process. Brad felt that there was a need to challenge both the user and the designer, and not be afraid of failure or conflict.
It is also important to learn quickly on-the-fly. The textbook type preparation is a critical base of knowledge to have, but it should evolve as you absorb the practical learning when interacting with the participants. This is a general principle of modern day software development, where change comes fast and may be disruptive to the way you work.
Post-study feedback
Following the study, we created actionable items in the form of user stories to bring these insights to life, and after making changes, Brad shared screenshots with willing participants to get reactionary feedback and iterate based on the impressions it left.
Though Brad and I were successful collaborators, I identified few areas that could have been improved.
Data analysis (researcher only)
I was rushed for time to have a playback, so Brad quickly synthesized the findings. I should have involved him earlier in my data analysis, as it would have helped him to see his insights quicker.
Draft playback deck
Though we had regular Slack conversations where I shared the draft insights, Brad was a bit surprised in seeing the playback deck for the first time, specifically the content and reporting. I could have avoided this surprise by sharing few examples of my playbacks much earlier in the study.
Internal usability testing
As Brad was working on a new competitive feature, we did not want our competitors to learn about it. So, we had to reach out to IBMers to participate in the study, and it took some time for us to recruit the participants. Having a pool of IBMers for any study is always a good idea.
Expectations from developer
As I was preparing and progressing with the study plan and study sessions, I was only thinking of insights from a researcher’s perspective. I should have stopped and asked Brad about his expectation from the study as he was new to IBM and collaborating with me for the first time. This would have provided a better sense of collaboration across the study as a whole.
Final Thoughts
To summarize, we want to leave you with the three key takeaways that contributed to a successful study:
- Maintain a common understanding between designer and developer. Each role brings their own unique and important perspective of users.
- Decisions for actionable recommendations should be made together, keeping in mind the user needs and developer’s time availability.
- Even small changes made to the prototype are a win, as they demonstrate the value of human-centered design in the product.
We are happy to share that insights from the study made its way to the version 1.0 of the REXX capability in March 2021.
Below are a few snippets of social media and developer blogs announcing the GA release of the REXX capability. On the far right, there is a snippet of Brad’s blog sharing the GA release.
Accessibility Links:
