Building a Full-Stack Visual Debugger: The Team Story
I’m a runner. I love to run.
I love feeling the wind against my skin, my heart pounding in my chest as if it were about to explode; there is pain, and I feel exhausted but, at the same time I feel the exhilaration, a burst of energy I didn’t know I had. I keep running; I run toward the horizon. I keep putting one foot in front of the other; step by step, I beat my own limits. Step! Step! I have a purpose, a goal, and I can’t or won’t give up.
I know I’ve got this! You see, I love to set goals, for example, running a half-marathon in under two hours. In my mind, I can already visualize myself crossing the finish line and looking at the clock to see 01h59m59s. However, accomplishing that would take a long time and a lot of dedication. In a way, it’s very similar to goals for a project at work. For any project you wish to accomplish, you need to first understand the problem you want to solve, then set a goal. And when you have a goal, you need a plan!
Let Me Tell You a Story About a Debugging Dev
Speaking of goals and plans, let me tell you a story about this one project in particular that required a lot of focus, a plan. OutSystems users are used to building applications visually. They create actions and logic using arrows, boxes, etc. They design databases in only a few clicks. Our users already had the power of debugging server-side logic without ever having to look at any code. But then, along came mobile apps and suddenly, there were JavaScript, applications with client-side logic, offline capabilities, and native plugins (like GPS). Debugging server-side logic is easy for our users, but on the client side, it was a completely different game.
We wanted to provide the best user experience! We knew we could make the debugging experience much better. At that time, users had to open Chrome DevTools and then put on their super-geek-hats to inspect the generated code and look for errors. When I say “super-geeks,” I really, really mean it. We’re talking about code that is generated to be asynchronous, which is one of the most unpredictable things ever. And there’s more — when using a browser, we could not reproduce all the errors, because native apps and native plugins don’t run on the browser. The pain was real!
We wanted to take away that pain. And that’s the story I’m eager to share with you. A story about the project lifecycle for the Full-Stack Visual Debugger that would change and improve our lives and the lives of every mobile developer out there. Ambitious? Maybe, but we did it!
Client Side, Server Side, Debugging Vision
A project’s purpose is largely about addressing an issue; in this case, the issue was the pain of debugging mobile apps. We had a lot of user feedback, and they were all saying the same thing: debugging mobile applications is hard.
Addressing the pain of debugging mobile apps meant gathering the right people, keeping them motivated, meeting the deadlines, and accomplishing different milestones. Our goal was:
Develop an easy way to debug mobile apps in runtime and a consistent way of debugging both mobile and web responsive apps
We could easily have gotten drunk on the power of building this full-stack visual debugger thing. But we didn’t. Debugging is not a new concept. So, we did our homework, and that meant a lot of research.
There are already plenty of other development environments with this mobile debugging capability. We didn’t intend to reinvent the wheel. So we took a look at Brackets, Eclipse, Visual Studio, and Visual Studio Code to get some insights. We wanted to know which features they offered and how they handled things like adding a breakpoint, stepping over, stepping into, stop on all exceptions, etc. Besides the basic features, we also explored how they dealt with edge cases that lead to exceptions in the perfect developer experience. For instance, if the browser being used to debug the application happened to crash.
We didn’t stop there. We interviewed users, people that were already used to debugging client-side logic with this kind of development platform. This helped us understand what their debugger experience was like and where they struggled the most. What their needs were, what their pain was.
All this relevant information helped us build a vision based on a set of prioritized user stories. We visualized how the project would happen and we were confident with what we needed to do to implement the Full-Stack Visual Debugger. The last step was to create a proof of concept to confirm whether the idea that lived in our minds was feasible. Was it?
Milestones Around the Checkpoint
The Full-Stack Visual Debugger project was split into milestones. A milestone is a checkpoint where we deliver a set of user stories that, when combined, make it possible for the user to accomplish a specific task of the project goal. For example, to run a half marathon I would have to start with short distances and slowly build up my endurance, for example 5K, then 10K, then maybe 15K at a time. This would keep my progress in check. And, I would be closer to my goal, one milestone at a time.
While planning these milestones and keeping in mind our vision based on the research results, we already had some ideas about how the project should happen.
Our product architecture has several layers, and our R&D teams are split up in such a way that each team is responsible for a layer. In this project we would have to make changes to two layers:
- Development environment layer: The user should be able to debug their application visually.
- Application runtime layer: The applications should contain an API used by the development environment to debug the running application.
Looking at this, we were able to understand that the debugger project would need at least two teams to be responsible for these two layers. Plus, during development, we had to keep in mind the entire experience and all the documentation and the copy, so two other teams joined the project.
The Fantastic Four (Teams)
The anticipated moment had arrived: the start of our project! The first task was to extract all the information from our debugger expert’s brain. By learning and understanding the architecture, we could extend the existing architecture with new capabilities, while redesigning our house plans.
While a team was setting up the house remodel, preparing our development environment, another team was building the furniture (APIs that would be called by our development environment to debug the running application). We would use the APIs to furnish our rooms (debugger capabilities).
We also had a team planning what this furniture would look like inside our remodeled house. Along the way, we also wrote documentation and copy to ensure that our users would clearly understand the furniture’s purpose.
Because the APIs and the features were being created simultaneously by different teams, we tended to have issues on hold while we waited on work from other teams. To avoid lack of focus, we started planning ahead. We checked everything that we had to implement that could be affected by dependencies with the other teams. Then, we split the tasks by research/design and implementation phases.
Of course, a super flexible engineering culture where everyone is open-minded and ready to experiment new ways of working makes collaboration and trying a new approach a lot easier. It seriously increases the likelihood of getting an awesome result!
We are talking about teams that didn’t even sit near each other, so we had to figure out a way to be sure that we were all working toward the same goal. We had a weekly get-together on the couch where everyone could bring up challenges, upcoming tasks, and the work backlog. Even though we belonged to different teams, we worked together as one.
The Faces of a Debugger Project
This was a debut role for many of us: a big challenge!
These were the roles:
- Team captain: The goal is to empower autonomous, passionate, and productive software engineers to deliver a killer product.
- Product owner: The goal is to deliver the right feature with the right scope at the right time.
- System owner: The goal is for the large multi-disciplinary R&D team, through a correct product architecture, to deliver a product that is fast to evolve.
- Product designer: The goal is to deliver a product users will love at first sight and keep on loving forever.
All these people had to ensure the ultimate low-code experience. You see, while you are implementing a new feature, sometimes you are so focused on the code and you are so biased toward what you see everyday that you risk getting to the end of the project without a good, complete experience for your user.
The real test was having alpha users. This project would hugely affect developers, so we couldn’t sit in our corner, develop an entire feature for months and then knock on their doors and say “Here’s a gift! Surprise!”
We would give a very exclusive user group the privilege to access the basic functions and try out first hand the feature still under development. This would provide us real-time feedback, so we asked some internal teams that use our product to tell us what they thought from day one. We wanted to mitigate risk at all costs. Debugging mobile applications using our device was generating a lot of uncertainty since there was a factor (the device) that we couldn’t control. So we sent an email to the whole engineering team, asking them to use the new debugger on their devices (smartphones). With this, we were able to catch even more problems before the public release.
Debugging Client Side… in the Device
Alpha users started dropping by our desk; they thanked us for allowing them to debug their applications a lot faster and a lot more efficiently. However, they also complained that they weren’t able to debug an application in the browser when using native plugins, which was a scenario that we had planned to handle in a milestone toward the end of the project. The browser was putting limitations on the user experience, and we realized that this issue was even more important to our users than we had previously anticipated.
It was now clear to us that to deliver the game-changing debugging capabilities that would improve the whole user experience, we would need to prioritize that debugging milestone (debugging an application directly on a device where the native plugins are available). Then, we re-prioritized our milestones to include these debugging capabilities in the first public release. Re-prioritizing milestones was not a simple process; we had to make tough decisions regarding features that we had planned and the new features that we had decided to prioritize. A lot of effort was spent planning a set of features that had the values we needed to remove the user experience limitations as highlighted by our alpha users.
When planning the milestones and when choosing what to include or exclude, it was essential that we not lose sight of how the end solution was going to look and whether it would take away the pain.
In the end, our alpha users were able to debug client side logic on their devices! This was life-changing for us and for our users. A breakthrough for our user experience! Without getting into too much technical detail, but just so you understand how much this meant to them, they even bought us donuts to thank us. It was awesome! Happy users -> Happy (Fat) Team!
As we were approaching the end of the project, we were proud of what we had achieved. With the Full-Stack Visual Debugger, what would usually take 3 hours now takes 5 minutes, and our users now have answers to their questions. When the project was released, we all celebrated together; it was a very rewarding moment for the team. And knowing that we made our users dreams come true felt beyond amazing. This is what engineering strives for at OutSystems!
Dreams come true.
Follow up with the Full-Stack Visual Debugger: One Implementation At a Time.
The ideas shared in this post resulted partially from the research project RADicalize, which was supported by the EU and PT2020.
