Computers Are Hard: app performance with Jeff Fritz

Published in

Computers Are Hard

11 min readSep 27, 2020

If the next person comes in and they can’t understand your code, it doesn’t matter how good it is.

Illustration showing a race car competing with an old car. — App performance with Jeff Fritz. Illustration by Gabi Krakowska.

Everyone wants fast software. We all know the agony of operating a touch screen that’s slow to respond, or trying to play a shooter with requirements way above what our PCs can handle. We hate loading screens, animated hourglasses, and spinning beach balls. There’s plenty of research into the psychology of human-computer interaction that boils down to this: if the app or website doesn’t respond immediately, we get frustrated, bored, or both.

But a computer has only so much processing power, memory, and storage. And it can have many applications competing for these resources. As I’m writing this, I have 25 processes open just in Chrome. There’s about 150 others, keeping Windows and a bunch of apps running. They’re using 35% of my PC’s memory and I’m not really even doing anything.

How do software engineers operate in these circumstances, where everyone wants every app to load immediately and run smoothly but at the same time not use too much RAM or processing power? How do they cope with never ending demands for better performance? I talked about this with Jeff Fritz. Jeff works at Microsoft as a Program Manager on the team responsible for ASP.NET, an open source web development framework. He walked me through how he thinks about and works on writing performant software.

Wojtek Borowicz: How do you define app performance? Is it how fast it loads and runs? How efficient it is with resource consumption? How stable?

Jeff Fritz: I look at performance as a mashup of all those things. If there are memory leaks, if there are issues with processor utilization, that application or system is going to crash sooner than later because it’s gonna run out of resources. It’s not so much a question of how much processor or memory is being used but whether the application is making the best use of them. It’s one thing to say ‘oh yeah, I’ve got very small utilization of processor and memory’. Well, if you didn’t need a lot of processor and memory, who cares? But if your application is really big and it does a lot of heavy processing, it’s doing some artificial intelligence, modelling, and calculation, well… it needs a lot of processor. Is it using that efficiently?

And finally, there is the user’s perception. If the time from when I launch an application to when it starts responding is very long, that’s terrible performance. There are ways to hide that. Push it to background threads or put up spinners and waiting indicators to let folks know that ‘I’m doing some work in the background, you can continue doing something over here while I finish over there’.

So part of an app’s performance is in the user’s perception and not just in how efficiently it utilizes the system’s resources?

If I had to rate those types of performance measurement, I would put them in that order: stability, the time and ability for the user to interact, and the efficiency of the utilization of processor and memory

If stability drops quickly, that’s terrible performance. The application is not performing well because it keeps falling down. The user’s perception of performance is number two because okay, we’re stable, but is it returning to me properly? And third would be resource utilization.

Is there much difference between platforms? Is it more difficult to develop fast apps for Windows, Linux, or Mac OS? How about mobile?

I don’t think there is much difference in how you tune for performance between various platforms. You always want to figure out how you can use other threads and push resource-intensive things into background processes, while you’re still servicing the user. You see that same concern whether it’s Windows, Mac, or Linux. Whether it’s desktop, or server, or cloud, or mobile. It’s just a difference in how much resources you have available. When you’re on mobile, you have the phone’s processor. For IoT, you might only have a little processor on a Raspberry Pi. You’re on the cloud? You have terabytes of memory available, dozens of processors, each that has a handful of cores… and you’re still gonna run into very similar concepts.

Tricks come in when you have one thread. Maybe it’s JavaScript or Web Assembly inside a browser. You’re in a sandbox, in a very constrained environment. You can’t play games with background threads. There’s tricks you can pull but it’s still the same thing: where can you stash resources so the processor you have inside a web browser sandbox is able to properly respond and paint the screen in a quick manner?

THREADS
When you launch an app, it starts a process on your computer. You can look it up in Task Manager (Windows) or Activity Monitor (Mac). Processes request the CPU to perform instructions. A sequence of such instructions is called a thread and it’s a basic unit of CPU utilization.

Are there programming languages and frameworks that are inherently more performant than others?

I wouldn’t say that. But there are tools that allow you to interact with platforms at a lower level and act more directly with the processor, with memory, with network. If you can program with C++, then you have full access to every bit of the processor. To every register in memory. But you don’t really have garbage collection. You actually have so much control, that it’s not the right tool to build a website with and deploy something that’s gonna run in a browser and interact with much higher level languages like JavaScript and HTML.

There are tools that are better for some environments. There are some general purpose ones. You’re kind of forced into Objective-C and Swift if you want to compile for iOS. You have to be in a Java-like environment if you want to be on Android. JavaScript works in many places. But do you really want to write JavaScript to build a game? Are you gonna get the best performance out of that? Nah. Because in JavaScript, you don’t have control over memory utilization and processor. And you can run into issues based on the traits of the language. To each their own.

I like C#. It will compile and run everywhere and it’s really good for me to be productive. It improves my developer performance. Does that give me the best experience everywhere? No, I’m not gonna get the best performance in an application deployed to Android or iPhone. But I can take advantage of things that will speed up performance in those.

While we’re on the subject of different platforms. Is using Electron or Progressive Web Apps always going to be a performance trade-off versus building native apps?

There is a bit of a performance trade-off. There is a choice that you’re making. When you build Progressive Web Apps, you’re building a website that’s going to be able to be installed and run as a local application on Windows, Mac, Linux, iPhone, Android. Nice! But it’s a web application. You’re not going to be able to really touch and interact with sensors and devices. You can hit some Web APIs that will allow you to get to, say, GPS, cameras, and stuff. The little things. But you’re not going to be able to build high performance applications.

Electron is effectively a Progressive Web App that comes with its own browser. That being said, you can tune and really squeeze performance out of that. One of the most used Electron apps is Visual Studio Code and it’s tremendous with performance. I don’t work with the Visual Studio Code team but I see the delivery of their work. It’s a phenomenal editor that many, many people use and for an Electron app, it’s pretty impressive. But am I going to do artificial intelligence and machine learning inside an Electron app? Probably not. I might use an Electron app to view the output of a high-end application that is doing calculations around those things. Or maybe I’m a game developer and I’m rendering scenes for my next 3D game. I’m gonna have distributed computing running on big servers. Maybe they’re in the cloud, maybe in a local data center. Using an Electron to monitor and paint that performance on screen is a perfect use for that type of technology. But I wouldn’t have an Electron app rendering lighting and maps and ray tracing… you can do it, but you’re not gonna get the best performance from it.

ELECTRON
Electron is an open-source software development framework that allows web developers working with languages like JavaScript and HTML to package their software as apps for Mac, Windows, and Linux. It’s popular because it allows you to write one app and release it on multiple platforms. Among others, WhatsApp, Twitch, and Figma use Electron. But it has plenty of opponents, too, because of performance drawbacks of Electron apps compared to native software on each platform.

Hardware obviously plays a major role here. Apps will run smoother on a system with a faster CPU and plenty of RAM. But what about software? Can other apps running on the same system affect your program? If so, can you work around that?

This is where virtual machines and containers come into play. Some really intelligent folks have built these to be isolated across process boundaries, so that you can’t run into an issue where you’re significantly affected by other processes. You can be better isolated and be guaranteed that this container I’m running in has been allocated 4 GB of RAM or has been allocated X amount of processor cycles.

You see this on iPhone and Android. They will time out and sleep applications that you’re not currently using. Desktop operating systems (Mac OS, Windows 10, Linux) try to share the processor appropriately, so getting into more of a containerized application will prevent that side effect of stealing all the processor and knocking something down. But there are some applications where I want to own the entire processor. If I’m a big database, if I’m Microsoft SQL server, mySQL, or Postgres — I want to be the only application running on the system. You better believe I want 100% of the processor and I want all of the RAM.

If a customer reports your app uses too much RAM or runs slowly, how do you approach this?

That’s when you break out a profiling tool and something that will help you analyze memory utilization. That’s when you’re gonna run your application through a suite of integration tests. See if over the course of half hour or an hour memory utilization is trending up or it’s staying as close to flat as possible. All applications are at some point going to use more memory. The question is: how fast is acceptable?

Screenshot of Windows Task Manager with Chrome using 1.2 GB of RAM. — Turns out writing a blog can be quite memory-intensive.

Are those profiling and integration testing suites standard tools or is it something you need to develop yourself alongside your app?

You shouldn’t have to build a profiling tool. In my case, in the .NET world, Microsoft Visual Studio comes with a profiling tool. JetBrains makes a tool called dotTrace that will help you analyze your memory utilization as well. There’s a bunch of them out there from a number of different vendors.

Integration tests should be tests that you run as part of your quality assurance process. Before you ship something you should have some series of tests. A test can be as simple as a script that says explicitly ‘click this button’, ‘type this’, ‘click this’, and you make sure everything on the script works properly. Or maybe it’s an automation that folks know how to execute on any number of machines so you can parallel test everything very quickly.

The trick then is to measure. Measure what the processor or memory use was before the test ran and then measure them at the end and see what the delta is. If that change is acceptable, great! But if you started at 2% of the processor and at the end you’re using 10%, that’s a fivefold increase. I can’t say that’s unacceptable: if it’s a database system, if it’s MySQL or Postgres, and after scaling up and running a bunch of tests it’s only using 10% of the processor, that might be great. That might be tremendous. But if it’s a calculator app… what are you doing?

Screenshot from Chrome’s profiling tool, showing performance test results for facebook.com. — Testing Facebook’s web app’s performance in Chrome.

To what extent does an app have to be designed with performance in mind? Is this something you can always fix later or if you don’t take care of it early on, you’ll never pay back that debt?

This is one of the greatest challenges for developers. When do I start thinking about performance? And I hate to say it, but it depends. If you’re building a game, performance is critical. When folks are aiming their guns in a first-person shooter, they need to have great performance. Performance is a differentiator. But if you’re building a little website that’s gonna show information about upcoming conferences, brochureware, performance can be thought about later. You don’t need to design for performance up front. There are considerations you can make that will give you better performance without doing extra work. I can design a static website, so that when my user experiences it, they’re only getting static content, instead of generating pages on every request. Do you need to make that decision upfront? No, not all the time.

Are there limits to performance optimization? Is there a point where you say this is as fast and stable as it gets or is there always a margin to improve?

There is a school of thought that says you can always make something faster. But you will hit the limit of physics at some point and be like ‘We have X amount of threads running at the same time, I can only get those threads running so fast on this processor’. Also, as you’re optimizing things, you want to make sure they’re maintainable. If you engineer your system in a way that is incredibly ingenious and you squeeze that last bit of performance out of it in a very tricky way, that makes it now less maintainable. Is that a valid trade off? Is that valuable? When the next person comes in and they can’t understand your code, it doesn’t matter how good it is. They’re gonna say ‘I can’t understand this. I need to rewrite it’.

What common misconceptions do you notice among software engineers when it comes to performance?

There are misconceptions that there is one right way. That there is a golden framework or tool set that you can use and you will get the best performance available. That’s just not the case. You can get great performance in your applications with any number of tools, whether it’s C++, Java, PHP, Rust, Go, .NET tools… for whatever task it is, for whatever reason you’re building that application, you can get great performance with whatever tool you choose. There’s limits to how much you can get out of those things, but you can get great performance out of everything.