Member preview

JavaScript Frameworks, Performance Comparison

The ultimate performance battle between JavaScript frameworks

I thought it would be fun to use Stefan Krause’s benchmark tool to, only for the sake of competition, do some performance comparison between the most well-known front-end frameworks and UI libraries.

Disclaimer: You can use this article as a reference, but you must always run your own benchmarks and make decisions based on your project requirements. You may also want to look at other benchmarks like the real-world apps benchmarks. Also, be sure to check out Stefan’s latest report on his blog and his latest benchmark results.

Before reading the rest of the article, I would like you to keep something in mind. The majority of the UI libraries and frameworks out there are all fast and efficient enough. However, projects like Svelte stand out for me because of their approach and philosophy, for which I personally think they deserve more attention.

Now, with all that out of the way, grab a cup of coffee and enjoy the show!

Arena

All the benchmarks were executed on my MacBook Pro, with the following specs:

  • MacBook Pro (Retina, 15-inch, Mid 2015)
  • Processor: 2.2 GHz Intel Core i7
  • Memory: 16 GB 1600 MHz DDR3
  • Graphics: Intel Iris Pro 1536 MB
  • Browser: Google Chrome, Version 69.0.3497.100

Teams*

In our benchmark competition we have two teams, the frameworks and the libraries team. In the frameworks team (Team 1) we have:

In the UI libraries team (Team 2) we have:

The Battles

In this ultimate championship, each team is going to initially compete in their own teams. Next, to make things more interesting, the winners of each team are going to compete against each other. And finally, the top performers will compete against the all-time champion, PlainJS a.k.a VanillaJS.

Also, it’s worth mentioning that each team is going to compete in the following categories:

  1. DOM Manipulation
  2. Startup Time
  3. Memory Allocation

In order to have better results, each benchmark is carried out three times and the standard deviation for each result is recorded. Moreover, at the end of each teams’ battles, I will present a table summarizing the results for all the frameworks and libraries against the winner of the team.

At the end, for fun, I’ll also provide relative results for the most popular frameworks and libraries: Angular, Inferno, Preact, React, and Vue.

Team 1 Matches

The first round matches for Team 1 are listed below:

  • Angular vs Elm
  • AngularJS vs Choo
  • Marionette vs Mithril
  • Aurelia vs Ember

Battle 1: Angular vs Elm

Both Angular and Elm are great platforms for building applications. They are relatively larger than a framework, that’s why it made sense to have them compete with each other. Below is the summary of the results:

Running benchmarks for Angular 6.1.0 Optimized and Elm 0.19.0. The comparison is against Angular 6. The green color means the given framework performs better than Angular. The red color means that the given framework performs worse. The white color means that the benchmarking tool couldn’t draw a good conclusion. Also note that if the numbers are the same, but the colors are not, that means that the standard deviation calculation will determine the relative result.

As you can see from the results above, Angular performs much better in the DOM manipulation category. However, in the Startup Metrics and the Memory category Elm performs better overall. With that, Elm wins this close battle with a score of 2 to 1.

Elm wins Against Angular 6 Optimized

Battle 2: AngularJS vs Choo

Moving on to the next battle, we have AngularJS vs Choo. AngularJS has been around for a while, and now it’s on version 1.7.1. Moreover, as of July 1, 2018 it has entered a 3 year Long Term Support period. Choo is relatively newer framework with lighter size and a focus on developer’s productivity. Below is the summary of the results of the battle:

Benchmark results for Choo and AngularJS. The comparison is against Choo. The green color means that the given framework performs better and the red color means that the given framework performs worse than Choo.

As you can see, AngularJS overall performs better in the DOM Manipulation category. However, Choo is the clear winner in the other two categories, making Choo the winner with a score of 2 to 1.

Choo wins against AngularJS

Battle 3: Mithril vs Marionette

Both Marionette and Mithril are very practical frameworks and I thought it would be interesting to benchmark them against each other. Below is the summary of the results:

The comparison is against Mithril.

As you can see, Marionette clearly wins in the DOM Manipulation category. However, it loses in all the rows of the Startup Metrics category. And in the Memory Allocation category Marionette wins overall. And with that, Marionette wins with a score of 2 to 1.

Marionette wins, 2 to 1.

Battle 4: Ember vs Aurelia

In the final battle of round 1, we are going to have Ember compete against Aurelia. Both Ember and Aurelia are great frameworks with similar philosophies, and that’s why I thought it would be interesting to have them compete with each other. Below is the summary of the results:

Comparison against Aurelia

Looking the tables above, Aurelia wins in all three categories, making it the clear winner.

Aurelia wins against Ember, 3 to 0

Now, we have reached the end of round 1. Below are the winners of round 1:

  • Elm
  • Choo
  • Marionette
  • Aurelia

In the next round we are going to have the above competitors compete each other to find the winner of Team 1.

Team 1, Round 2

In this round, we are going to have the following matches:

  1. Elm vs Choo
  2. Aurelia vs Marionette

Round 2, Battle 1: Elms vs Choo

The following table summarizes the results:

Elm vs Choo. The values in the tables are compared to Elm.

Looking at the table above, we can see Choo clearly looses to Elm in the DOM category. However, it wins against Elm in the Memory category. Now, in the Startup category, Elm wins in 3 rows making it the winner.

Elm wins against Choo

Battle 2: Aurelia vs Marionette

The following table summarizes the results:

Marionette vs Aurelia. The values are compared against Aurelia.

Looking at the results above we can clearly see Marionette wins in all three categories making it the winner against Aurelia.

Marionette wins against Aurelia

Team 1, Final Round

In the final round we have Marionette compete against Elm. First let’s look the absolute values:

Elm vs Marionette absolute values with standard deviation

Looking at the absolute values above we can see that Marionette wins in the DOM category. But Marionette loses in the Startup category against Elm. And in the Memory category Marionette wins with a slight advantage over Elm.

Now let’s take a look at the comparison tables:

Marionette vs Elm, comparison table. Comparing values to Elm

Looking at the values above we can see that Marionette wins in the DOM and the Memory categories, but looses in the Startup category. But overall, Marionette wins against Elm.

Marionette wins against Elm

Now that we have a winner, let’s look at comparison tables of all the competitors against the winner, Marionette. Let’s start by looking at the DOM manipulation results:

Marionette vs all the members of Team 1. The values are compared to Marionette.

Looking at the values above we can clearly see that Angular 6 Optimized is the only competitor that clearly wins against Marionette. Now let’s look at the startup time comparisons:

Comparing startup times. Marionette vs all the competitors in Team 1

Looking at the table above we can see that Mithril and Elm perform better than Marionette. Finally, let’s look at the Memory Allocation comparison:

Memory comparison, Marionette vs all the members of Team 1

Looking at the table above we can see that Choo is the only competitor that performs better than Marionette in terms of memory allocation. Now, with that Marionette wins in team 1.

Marionette wins in Team 1

Team 2 Matches

The first round of matches for Team 2 are listed below:

  • React vs Vue
  • Preact vs Inferno
  • Svelte vs Redom
  • Bobril vs Maquette

Battle 1: React vs Vue

Both React and Vue are great libraries for creating component-based user interfaces. The “React vs Vue” question is asked so many times that I figured I’ll attempt to answer it my mighty tables :)

React vs Vue, all the values are relative to React.

Looking at the values above we can see that both React and Vue are very similar in the Startup and the Memory category. But because this is a competition we are going to take everything into account. Considering that, Vue wins in the Startup and the Memory categories. Now looking at the DOM category it looks more like a draw with a slight advantage towards Vue. And again since this is a competition I’m going to have to pick Vue as the winner in this very close matchup. Having said that, looking at the values above it’s fair to say that the “React vs Vue” question is kind of irrelevant in terms of performance because they are very close to each other.

Vue wins against React, very close match

Battle 2: Preact vs Inferno

Both Preact and Inferno are very high performant and lightweight UI libraries. Because of that I thought it would be interesting to have them compete with each other:

Preact vs Infero, all the values are relative to Preact

Looking at the table above we can clearly see Inferno wins in the DOM and the Memory categories. Now, in the Startup category Preact has a slight advantage but overall Inferno is the clear winner.

Inferno wins against Preact

Battle 3: Svelte vs Redom

It was challenging to decide who should Svelte compete against. That’s because Svelte defines a new category of libraries that are more like compilers than libraries. However, I thought it would be fun make Svelte compete against a high performant and focused UI library like Redom.

Svelte vs Redom, all the values are relative to Redom.

Looking at the values we can see that Redom has an advantage in the DOM category. But, Svelte wins in the Memory category. Now looking at the Startup category we can see that they are very close. And overall it’s really hard to pick a winner. But again because this is a competition I have to pick Redom as the winner because it has a slight advantage in the Startup category.

Redom wins against Svelte in a very close match

Battle 4: Bobril vs Maquette

Maquette is a very low-level virtual DOM implementation. And Bobril is a component-based UI library that uses virtual DOM and competes with React. So I thought it would be interesting to see how Bobril stacks up against Maquette. Below is the summary of the results:

Bobril vs Maquette, all the values are relative to Bobril

Looking at the values above we can see that Maquette loses to Bobril in the DOM manipulation category. But in the Startup category they are very close, with Maquette having slight advantage. And finally in the Memory category, it’s fair to say that Maquette wins against Bobril. Overall, we can conclude that Maquette wins with slight advantage in the Startup category.

Maquette wins against Bobril

Now, we have reached the end of round 1. Below are the winners of round 1:

  • Vue
  • Inferno
  • Redom
  • Maquette

In the next round we are going to have the above competitors compete each other to find the winner of Team 2.

Team 2, Round 2

In this round, we are going to have the following matches:

  1. Vue vs Inferno
  2. Redom vs Maquette

Round 2, Battle 1: Vue vs Inferno

The following table summarizes the results:

Inferno vs Vue, the values are relative to Vue

Looking at the values above it is clear that Inferno is the winner in all the three categories.

Inferno wins against Vue

Battle 2: Redom vs Maquett

The following table summarizes the results:

Redom vs Maquette, all the values are relative to Redom

Looking at the tables above we can see that Redom clearly wins in the DOM category. In the Startup category they are very close with Redom having a slight advantage in terms of size. And finally in the Memory category they are also very close to each other but Redom has a slight advantage. Overall, it’s fair to say that Redom wins since it has considerable better performance in the DOM category.

Redom wins against Maquette

Team 2, Final Round

In the final round we have Inferno competing against Redom. First let’s look the absolute values:

Inferno vs Redom absolute values.

Looking at the absolute values above we can see that Inferno is the clear winner in DOM category. Now, looking at the Startup category we can see that Redom has the advantage. And also in the Memory category Redom overall has the advantage making Redom the winner. Now let’s look at the comparative tables and see if that’s still true:

Redom vs Inferno, all the values are relative to Inferno

Again we can see that Inferno is the clear winner in the DOM category, but Redom has a slight advantage in the Startup and the Memory categories. Now, it’s really hard to pick the winner, because if we weight the DOM category more, Inferno is the clear winner. Otherwise, overall we can say Redom wins because of it’s slight advantage in the other two categories.

Redom wins against Inferno overall

Now let’s look at comparison tables of all the competitors against the winner, Redom. Let’s start by looking at the DOM manipulation results:

Redom vs the rest of Team 2 in the DOM category. All the values are relative to Redom.

Looking at the values above, as expected, Inferno is the clear winner in the DOM Manipulation category. Now, let’s look at the relative values in the Startup category:

Redom vs the rest of Team 2 in the Startup category. All the values are relative to Redom

Looking at the values above we can see that they all have a very fast Bootup time, except Vue and React. In the interactive row Redom wins against all the other competitors in Team 2. In the Main Thread Maquette has an advantage over Redom. And finally In the Total Byte Weight row Svelte has a slight advantage over Redom.

Now let’s look at the relative values for the Memory Allocation category:

Redom vs the rest of Team 2 in the Memory category. All the values are relative to Redom.

Looking a the values above we can see that Svelte has a clear advantage over Redom. But Redom wins overall against all the other Team 2 members in the Memory Allocation category. Even though Inferno and Maquette have better memory performance in two areas. And with that Redom is the winner of Team 2.

Redom wins in Team 2

Winners Battles

In this section we are going to look at the results of the battle between Marionette, Team 1 winner, and Redom, the winner of Team 2. Note that comparing Redom against Marionette is like comparing apples to oranges. But, let’s do it for fun anyways ;)

Let’s start by looking at the absolute values (with standard deviation):

Redom vs Marionette

Looking at the values above we can see that both Redom and Marionette perform well in all the three categories against each other. Except Marionette has a slower Bootup Time and it’s heavier in size, which is expected.

Now, let’s look at the comparison tables:

Redom vs Marionette, relative values. All the values are relative to Redom.

Looking at the values above we can see that Redom, as expected wins all the three categories. But again bear in mind that comparing Redom with Marionette is like comparing apples to oranges. But the important thing to note here is that even though Redom wins this battle, Marionette does an amazing job coming very close to Redom which is a very light-weight and focused UI library.

Battle Against All-time Champion

In this section the top performers are going to battle against the all-time champion, Plain JavaScript. Below is the line up:

  • Redom
  • Inferno
  • Svelte
  • Vue
  • React
  • Preact
  • Marionette
  • Elm
  • Choo
  • Angular 6

First, let’s look at the absolute values for the DOM Manipulation category:

Plain JavaScript compared to top performers. Greenish color means closer to Plain JavaScript, and redish color means worse than Plain JavaScript

Looking at the values above we can see that most of our competitors are in the green areas. The one columns that has a lot of reds is Choo, but apart from that the rest perform pretty well compared to Plain JavaScript. Now let’s look at the Startup values:

Plain JavaScript compared to top performers. The values are sorted in terms of the Total Byte Weight, from lightest to heaviest.

Looking at the values above we can see that overall the competitors are in the greenish region compared to Plain JavaScript. We need to point out however in the Script Bootup Time row, Choo, Vue, Marionette, React, and Angular are slower and that’s expected because they are larger than UI libraries.

Next let’s look at the values in the Memory Allocation category:

Plain JavaScript compared to top performers. The values are sorted in terms of the memory usage after adding 1000 rows.

Looking at the values above we see more yellows than greens, which is again to be expected. But it’s worth mentioning that Choo, Svelte, and Redom have more greens than yellows which make them pretty good in the UI category. Looking at the frameworks we can see that Marionette has less yellows compared to the rest. And React seems to have most of the orange colors.

And finally let’s look at the comparison tables:

Comparing Plain JavaScript with the top performs in the DOM category. The values are sorted in terms of the Geometric Mean

Looking at the values above we can see that Inferno has an impressive performance followed by Angular 6 Optimized. But Inferno definitely wins in tis category. Next, let’s look at the relative Startup values:

Comparing Plain JavaScript to the top performers in the Startup category. All the values are sorted in terms of the Total Byte Weight

Looking at the values above we can see that Svelte is in the second place after Plain JavaScript in terms of total Byte Weight. And in the Bootup Time we have Svelte, Redom, Preact, Elm, and Inferno all at 16ms. In the Consistently Interactive row we have Redom very close to Plain JavaScript.

Next, let’s look at the relative values in the Memory category:

Comparing Plain JavaScript to top performers in terms of Memory Allocation. The results are sorted in terms of the memory usage after adding 1000 rows.

Looking at the values above we can clearly see that Plain JavaScript wins against all the top performers in the Memory category.

Considering all the results above against Plain JavaScript we can reach the following conclusions:

  • Plain JavaScript performs better in terms of memory usage compared to all the top performers.
  • In the DOM manipulation performance, Inferno and Angular 6 Optimzed overall perform better compared to Plain JavaScript.
  • In the Startup category, Svelte, Redom, Preact, Elm, and Inferno have very fast bootup time similar to Plain JavaScript
  • In terms of total byte weight, Svelte, Redom, Preact are very similar to Plain JavaScript in the UI libraries category. In the frameworks category, we have Elm, and Choo in the greenish region relative to Plain JavaScript.
  • In terms of being consistently interactive and main thread work cost, it’s not very easy to draw conclusions by looking at the tables. But we can say that Redom is closest to Plain JavaScript performance.

Angular vs Vue vs React vs Preact vs Inferno

As promised, I’m going to include the comparison tables for the famous question:

Angular vs Vue vs React vs Preact vs Inferno

I’m going to present the tables, but I’ll leave it to the readers to draw their own conclusions. Starting with the raw values we have:

All the vales are sorted in terms of the total byte weight

Now let’s look at the relative tables. First up we have values relative to React:

Values compared to React. Green means better than React, red means worse than react. All the framework are sorted in terms of the total byte weight.

Next we have values relative to Vue:

Values compared to Vue. Green means better than Vue, red means worse than Vue. All the framework are sorted in terms of the total byte weight.

Next up, we have values compared to Preact:

Values compared to Preact. Green means better than Preact, red means worse than Preact. All the framework are sorted in terms of the total byte weight.

Next we have values compared to Angular 6 (Optimized):

Values compared to Angular. Green means better than Angular, red means worse than Angular. All the framework are sorted in terms of the total byte weight.

And finally, we have values relative to Inferno:

Values compared to Inferno. Green means better than Inferno, red means worse than Inferno. All the framework are sorted in terms of the total byte weight.

Conclusion

Performance benchmarking is a hot topic and talking about it is challenging. In this article however, I attempted to provide a starting point for those who are interested in the topic. As mentioned before, you should always run your own benchmarks, create prototypes and evaluate options based on your project requirements. Let me know what you think, I would love to hear your thoughts.


Appendix 1

For reference, below is the list of all the frameworks and libraries mentioned in the article, sorted in terms of stars on Github from highest to lowest:

  1. Vue: 115,033
  2. React: 112,075
  3. AngularJS: 59,139
  4. Angular: 41,049
  5. Preact: 20,192
  6. Ember: 20,043
  7. Inferno: 12,939
  8. Aurelia: 10,641
  9. Mithril: 9,543
  10. Knockout: 9,146
  11. Svelte: 8,069
  12. Marionette: 7,171
  13. Choo: 5,676
  14. Redom: 1,464
  15. Maquette: 657
  16. Apprun: 439
  17. Bobril: 300
  18. Surplus: 299
  19. ivi: 181 (TypeScript)

Appendix 2

Here are some more screenshots taken directly from Stefan’s interactive tool that includes more results. All the values are sorted in terms of the total byte size:

Footnotes:

  1. It was kind of challenging to categorize the frameworks and libraries into separate teams. The simplest solution was to simply say frameworks versus libraries. For example, I couldn’t really decide to put Vue in Team 1 or Team 2. I finally decided to include it in Team 2.
  2. I used Angular v6.1.0 Optimized for all the benchmarks.
  3. Svelte is not really a UI library, but rather it’s a compiler. For simplicity sake I grouped Svelte under the UI libraries.