Why profiling should be part of regular software development workflow
And how to make it so
Recently, I had the great opportunity to speak at WebDevCon 2024 in Amsterdam. My talk, “What browser profilers teach us about Javascript,” drew inspiration from the many things I’ve learned by working on and working with profilers.
Like many, performance optimization was the reason I got initiated into the space of profiling. However, after joining Datadog’s profiling team, I’ve come to realize that the use case of profiling extends far beyond performance optimization. Like I’ve written here, profiling can often help its users understand how a language works.
To me, one of the biggest impacts profiling can have is revealing the unknown unknowns to its users.
Unknown Unknowns: things we are neither aware of nor understand.
As an example, many things affect frontend performance. I know some of these things: long tasks can make applications feel laggy, certain bad coding practices lead to unnecessary rendering in React. However, one thing I was neither aware of nor understood is the role that garbage collection plays in performance. Did you know that in the context of running Javascript in the browser, garbage collection blocks the main thread?
If you had taken a look at some browser profilers, you would have learned it.
Look at all the garbage collection activities during a 3.5s long profile — during which the main thread was blocked.
For more details on what these browser profilers can teach us about Javascript memory management, read this blog post, which served as an inspiration for my talk.
Garbage collection went from a thing I did not know I needed to care about to something I now know is important to keep an eye on when looking at performance issues.
Despite the capabilities of good profiling software, profiling has a reputation for being hard. Many friends and colleagues I know in the industry have struggled or have seen others struggle with this tool. Because of this, profiling is a tool many believe only performance experts can use.
Over the last year, I’ve stopped viewing profiling as a tool for performance experts. Instead, I started seeing it as a “table of contents” of a very useful manual. In this case, the manual of how a language works, or what causes performance issues.
A good profiler is akin to a “table of contents” of an useful manual.
In my very first week alone after joining Datadog, I learned a ton about CPU optimization from just looking at the Datadog’s Continuous Profiler UI.
The Datadog Continuous Profiler reveals to its users things that can contribute to high CPU usage.
High garbage collection load, high regex compilation and high primitive value boxing are all signs of problematic code. If you were to click on one of these cards, you’d learn more about it.
From these profiling insights alone, I learn about some low hanging fruits of performance optimization. This is before even getting into what Flame Graphs and other visualizations can teach us, which is a lot.
Even though these profilers that the profiling team at Datadog work on are all backend profilers, and I myself am a frontend engineer, much of the knowledge that profilers can impart to its users transcends languages and runtimes.
For instance, I previously wrote a blog post on how the Chrome DevTools Memory tab works.
In that blog post, I went into details about live heap, allocation sampling, and allocation instrumentation on timeline. How to interpret and make use of heap profiles or allocated memory profile are all concepts I learned by looking at Flame Graphs of the various runtimes Datadog supports like Go, Ruby, etc.
I’ve gained valuable insights into frontend performance nuances, like the impact of garbage collection, through my experience on the profiling team at Datadog, despite focusing primarily on the UI for backend profilers. This goes to show the significant advantage and efficacy of integrating profiling into your daily workflow.
However, despite the capabilities of good profiling softwares, profiling has a reputation for being something only performance experts can do. Many engineers, myself included, struggled with it when we first got started. Part of the reason comes from the complexity of the software themselves, or ineffective usage of visualization tools. For example, the visualization most engineers are familiar with, the Flame Graph, is not always the best visualization for solving problems.
It’s easier to change ourselves than it is to change the world. Even though good profiling software can be complex, I truly believe that seeing profiling as a “table of contents” of an useful manual for how things work is a good step to overcoming this high barrier to entry.
