Magento 2.4 CE vs Aero Commerce Performance Comparison

Let's check the performance of the beast eCommerce Magento 2 and the modern eCommerce project.

In the e-commerce industry, nearly half of the consumers always expect a website to load in 2 seconds or less. About forty percent of the online shopper will abandon a web page that takes longer than 3s to load!

Amazon has found that every 100ms of latency cost them 1% in sales.

Google has found an extra 0.5 seconds in search page generation time dropped traffic by 20%. A broker could lose $4 million in revenues per millisecond if their electronic trading platform is 5 milliseconds behind the competition.

When AutoAnything reduced page load time by half, they saw a boost of 12% to 13% in sales.

Retailer Furniture Village audited their site speed and developed a plan to address the problems they found, leading to a 20% reduction in page load time and a 10% increase in conversion rate.

We will test the next steps:

1. Simple Product Page
2. Configurable Page
3. Category Page
4. Search T-shirt
5. CMS page
6. Add to cart AJAX
7. Cart Page
8. API Get Category Page

We will test both Servers Framework performance and Google Page Speed Insight Frontend Performance of the core demo sites.

Test made with Apache Benchmark tool from the same server to avoid network overhead.

Also, we will check the sites under the load

Magento 2

Magento 2 test environment C5.large AWS, PHP 7.4.11, Nginx, Elastic Search Luma Them + Sample Data.

No needs to describe what Magento 2 is. Everybody knows that it is a well-advertised platform owned by Adobe giant.

Generated optimized autoload files containing 27253 classes. Codebase 4 253 069 lines of PHP code.

Note: Magento 2 CE is 20–30% faster than paid one Magento Enterprise/Commerce.

Aero Commerce

C5.large, PHP 7.4.11, Nginx, Elastic Search, default out of the box configuration without any additional improvements.

Generated optimized autoload files containing 6232 classes. Codebase 552 882 lines of PHP code.

Aero Commerce is a performance-first e-commerce platform based on the modern Laravel PHP framework, designed to better suit the needs of retailers that are serious about eCommerce.

Magento Master Willem Wigman about Laravel ecosystem

Laravel ecosystem is really great! I have so much fun.

Willem Wigman (Magento Master)

Performance test:

Simple product

Magento:

/fusion-backpack.html

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   255  258   2.2    258     262
Waiting:      254  257   2.2    257     261
Total:        255  258   2.2    258     263

Result: 257ms

Aero:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    2   0.0      2       2
Processing:   109  110   0.8    110     112
Waiting:      109  110   0.8    110     112
Total:        111  112   0.8    112     114

Result: 110ms

2. Configurable Products Performance

Magento 5 x 3

/hero-hoodie.html

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   286  291   6.7    289     305
Waiting:      284  290   6.9    288     304
Total:        286  292   6.7    289     305

Result: 288ms

Aero 4 x 5

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    2   0.0      2       2
Processing:   118  119   0.6    119     120
Waiting:      117  118   0.6    118     119
Total:        120  120   0.6    121     121

Result: 118ms

3. Category page

Magento (24 products per page-default):

/women/tops-women/jackets-women.html?product_list_limit=24

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   280  300  15.9    300     325
Waiting:      279  298  15.9    297     324
Total:        281  300  15.9    300     325

Result: 297ms

Aero (24 products per page — default):

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    2   0.1      2       2
Processing:   153  163   9.0    162     183
Waiting:      152  162   9.1    162     182
Total:        154  165   9.0    164     184

Result: 162ms

4. Search “T-shirt”

Magento 5 results:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   256  294  41.3    286     392
Waiting:      252  289  41.8    281     388
Total:        256  294  41.3    287     392

Result: 281ms. However, the worst result is 388ms. I think to correctly review performance we should use the slowest result. However, it is in the next test. In this test, we will use an average time in favor of Magento 2.

Aero 7 results:

Aero Results is more nice has layered navigation vs Magento 2.4.0. Even ElasticSearch can’t help Magento 2.4.+ to help fast and good test.

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    2   0.2      2       2
Processing:   141  154  14.0    150     187
Waiting:      140  153  14.0    149     186
Total:        142  156  14.0    151     189

Result: 149ms the worst result is 186ms

5. CMS Page

Magento 2:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   160  162   1.5    162     165
Waiting:      159  161   1.1    161     162
Total:        161  162   1.5    162     166

Result: 151ms

Aero:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    2   0.0      2       2
Processing:    14   14   0.8     14      16
Waiting:       13   14   0.8     14      16
Total:         15   16   0.8     15      17

Result: 14ms

Cms page is the lightest possible eCommerce Functionality functionality without any heavy Product/CAtegory collection models. . It is something like “Hello World” from the database. ‘Hello World’ Benchmark — mini benchmark, comparing the performance HTTP ‘hello world’ on different freamworks. We can see that Magento 2 stack is 10 times slower than modern php app Laravel stack.

Laravel is a great example of a PHP ecosystem that is innovating and thriving in an incredible pace. Hyva(Magento Frontend)

Magento 2 stack and Laravel(Aero Commerce) stack popularity according google trends:

6. Add to cart AJAX

Magento:

Result: 255ms

Aero:

Result: 55ms

7. Cart Page

Magento:

Result: 450ms

Aero:

Result 51ms

And don't forget that the regular Magento 2 page also needs additional requests to slow backend to fetch additional data(customer, log visitor, dynamic sections, banners, shipping and tax calculation,etc.) to be finished. Just a simple request “/customer/section/load/?sections=compare-products%2Cmessages&force_new_section_timestamp=false&_=1602575313245" takes 150ms. The checkout page needs around 10 slow requests.

8. API Get category Page

Magento 2:

Graph QL get category data by ID

SSH POST Curl request:
time curl --request POST   --url https://venia.magento.com/graphql   --header 'cache-control: no-cache'   --header 'content-type: application/json'   --header 'postman-token: 3d74cd1b-eb7f-a191-d115-910f4c0bfa09'   --data '{"operationName":"category","variables":{"currentPage":1,"id":19,"filters":{"category_id":{"eq":"19"}},"onServer":false,"pageSize":6,"sort":{"relevance":"DESC"}},"query":"query category($id: Int!, $pageSize: Int!, $currentPage: Int!, $onServer: Boolean!, $filters: ProductAttributeFilterInput!, $sort: ProductAttributeSortInput) {\n  category(id: $id) {\n    id\n    description\n    name\n    product_count\n    meta_title @include(if: $onServer)\n    meta_keywords @include(if: $onServer)\n    meta_description\n    __typename\n  }\n  products(pageSize: $pageSize, currentPage: $currentPage, filter: $filters, sort: $sort) {\n    items {\n      __typename\n      description {\n        html\n        __typename\n      }\n      id\n      media_gallery_entries {\n        id\n        label\n        position\n        disabled\n        file\n        __typename\n      }\n      meta_title @include(if: $onServer)\n      meta_keyword @include(if: $onServer)\n      meta_description\n      name\n      price {\n        regularPrice {\n          amount {\n            currency\n            value\n            __typename\n          }\n          __typename\n        }\n        __typename\n      }\n      sku\n      small_image {\n        url\n        __typename\n      }\n      url_key\n      ... on ConfigurableProduct {\n        configurable_options {\n          attribute_code\n          attribute_id\n          id\n          label\n          values {\n            default_label\n            label\n            store_label\n            use_default_value\n            value_index\n            swatch_data {\n              ... on ImageSwatchData {\n                thumbnail\n                __typename\n              }\n              value\n              __typename\n            }\n            __typename\n          }\n          __typename\n        }\n        variants {\n          attributes {\n            code\n            value_index\n            __typename\n          }\n          product {\n            id\n            media_gallery_entries {\n              id\n              disabled\n              file\n              label\n              position\n              __typename\n            }\n            sku\n            stock_status\n            __typename\n          }\n          __typename\n        }\n        __typename\n      }\n    }\n    page_info {\n      total_pages\n      __typename\n    }\n    total_count\n    __typename\n  }\n}\n"}'

Result: 337ms

If to test on the same site hosted on Magento Cloud result is

Result: 693ms.

In this Magento 2 test, it is fast because of the proper AWS environment. For other hostings and Magento, Cloud results are x3 slower. Also if to install several Best 3-d party modules regular performance of the Magento pages is not faster than 1000ms.

To check real Magento 2 performance on Magento Cloud read this post:

Magento 2 performance of the Helly Hansen eCommerce hosted on the Magento Commerce Cloud

Aero Commerce:

just simple HTTP request: https://yehor.tqt.io/mens/t-shirts?json

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    2   0.3      2       3
Processing:    33   38   4.0     36      44
Waiting:       33   38   4.0     36      44
Total:         34   40   4.0     39      46

Result: 36ms

Performance Pivot table and Charts

Stacked column chart:

Magento 2.4.o vs Aero Commerce Performance

A stacked column chart allows part-to-whole comparisons. In a stacked column chart, data series are stacked one on top of the other in vertical columns. Stacked column charts can show performance differences because it’s easy to compare total column lengths. Less is better.

Standard chart:

Magento 2.4 vs Modern eCommerce Performance

Lighthouse PageSpeed Insights Performance Product page:

Magento 2:

source: https://twitter.com/hashtag/memento

Broken Magento's default JS Bundling is disabled to improve performance.

Mobile Performance:

Lab Data:

First Contentful Paint: 2.2 s

First Contentful Paint marks the time at which the first text or image is painted.

Speed Index: 18.1 s

Speed Index shows how quickly the contents of a page are visibly populated.

Largest Contentful Paint: 3.4 s

Largest Contentful Paint marks the time at which the largest text or image is painted.

Time to Interactive: 29.0 s

Time to interactive is the amount of time it takes for the page to become fully interactive.

Total Blocking Time: 200 ms

Sum of all time periods between FCP and Time to Interactive, when task length exceeded 50ms, expressed in milliseconds.

Cumulative Layout Shift: 0.439

Cumulative Layout Shift measures the movement of visible elements within the viewport.

Desktop Performance:

Lab Data:

First Contentful Paint: 0.7 s

Speed Index: 5.5 s

Largest Contentful Paint: 1.2 s

Time to Interactive: 10.0 s

Total Blocking Time: 0 ms

Cumulative Layout Shift: 0.086

Result: 56 Mobile and 68 Desktop

Aero Commerce:

All setting is default

Mobile Performance:

Lab Data:

First Contentful Paint: 1.0 s

Speed Index: 1.8 s

Largest Contentful Paint: 1.7 s

Time to Interactive: 2.5 s

Total Blocking Time: 90 ms

Cumulative Layout Shift: 0

Desktop Performance:

First Contentful Paint: 0.3 s

Speed Index: 0.6 s

Largest Contentful Paint: 0.7 s

Time to Interactive: 0.7 s

Total Blocking Time: 20 ms

Cumulative Layout Shift: 0

Aero Commerce frontend performance is awesome. Magento even can’t pass Web Vital requirements.

Web Vitals - quality signals key to delivering great UX on the web (https://web.dev/vitals). This extension measures the Core Web Vitals, providing instant feedback on loading, interactivity, and layout shift metrics.

Magento 2.4.o and Aero Commerce Performance Pivot table:

Page Speed Magento vs Aero (more is better)

Content Breakdown Magento 2 Product Page:

Aero Commerce Content Breakdown Product Page:

My suggestions to improve Aero Commerce performance:

I would like to suggest for the Aero commerce teem to improve Mobile blocking time performance by fixing these several issues:

<script>!(function(d){var b=d.getElementById(‘body-loading’);b.parentNode.removeChild(b)})(document);!(function(w){w.toggleClass=function(s,l){if(s.classList)s.classList.toggle(l);else{var a=s.className.split(“ “),e=a.indexOf(l);e>=0?a.splice(e,1):a.push(l),s.className=a.join(“ “)}};w.addClass=function(s,a){s.classList?s.classList.add(a):s.className+=” “+a};w.removeClass=function(s,e){s.classList?s.classList.remove(e):s.className=s.className.replace(new RegExp(“(^|\\b)”+e.split(“ “).join(“|”)+”(\\b|$)”,”gi”),” “)};})(window);</script>

Style Recalculation on mobile is really slow.

and next issue timer Fired event is to slow on mobile

and

client.js compilation is too slow. It is better to divide it into 3 chanks to avoid main thread blocking for a long time. Code splitting is a great way to improve its initial loading speed. Since a user doesn’t have to download all the code in one hit, they’ll be able to see and interact with the page sooner. This file loading VueJS however where it used on this page I hadn’t found. Maybe it is a redundant dependency on this page.

With Magento everything clear: it is to broken to be fixed. Even Mage Puck couldn't fix its bad frontend performance.

Magento 2 vs Shopware 6 peromance:

ShopWare 6.1 vs Magento 2.3.3 performance