Magento 2.4 CE vs Aero Commerce Performance Comparison

Yegor Shytikov
The Startup
Published in
9 min readOct 13, 2020

--

Let's check the performance of the beast eCommerce Magento 2 and the modern eCommerce project.

In the e-commerce industry, nearly half of the consumers always expect a website to load in 2 seconds or less. About forty percent of the online shopper will abandon a web page that takes longer than 3s to load!

Amazon has found that every 100ms of latency cost them 1% in sales.

Google has found an extra 0.5 seconds in search page generation time dropped traffic by 20%. A broker could lose $4 million in revenues per millisecond if their electronic trading platform is 5 milliseconds behind the competition.

When AutoAnything reduced page load time by half, they saw a boost of 12% to 13% in sales.

Retailer Furniture Village audited their site speed and developed a plan to address the problems they found, leading to a 20% reduction in page load time and a 10% increase in conversion rate.

We will test the next steps:

  1. Simple Product Page
  2. Configurable Page
  3. Category Page
  4. Search T-shirt
  5. CMS page
  6. Add to cart AJAX
  7. Cart Page
  8. API Get Category Page

We will test both Servers Framework performance and Google Page Speed Insight Frontend Performance of the core demo sites.

Test made with Apache Benchmark tool from the same server to avoid network overhead.

Also, we will check the sites under the load

Magento 2

Magento 2 test environment C5.large AWS, PHP 7.4.11, Nginx, Elastic Search Luma Them + Sample Data.

No needs to describe what Magento 2 is. Everybody knows that it is a well-advertised platform owned by Adobe giant.

Generated optimized autoload files containing 27253 classes. Codebase 4 253 069 lines of PHP code.

Note: Magento 2 CE is 20–30% faster than paid one Magento Enterprise/Commerce.

Aero Commerce

C5.large, PHP 7.4.11, Nginx, Elastic Search, default out of the box configuration without any additional improvements.

Generated optimized autoload files containing 6232 classes. Codebase 552 882 lines of PHP code.

Aero Commerce is a performance-first e-commerce platform based on the modern Laravel PHP framework, designed to better suit the needs of retailers that are serious about eCommerce.

Magento Master Willem Wigman about Laravel ecosystem

Laravel ecosystem is really great! I have so much fun.

Willem Wigman (Magento Master)

Performance test:

Simple product

Magento:

/fusion-backpack.html

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 255 258 2.2 258 262
Waiting: 254 257 2.2 257 261
Total: 255 258 2.2 258 263

Result: 257ms

Aero:

Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 2 0.0 2 2
Processing: 109 110 0.8 110 112
Waiting: 109 110 0.8 110 112
Total: 111 112 0.8 112 114

Result: 110ms

2. Configurable Products Performance

Magento 5 x 3

/hero-hoodie.html

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 286 291 6.7 289 305
Waiting: 284 290 6.9 288 304
Total: 286 292 6.7 289 305

Result: 288ms

Aero 4 x 5

Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 2 0.0 2 2
Processing: 118 119 0.6 119 120
Waiting: 117 118 0.6 118 119
Total: 120 120 0.6 121 121

Result: 118ms

3. Category page

Magento (24 products per page-default):

/women/tops-women/jackets-women.html?product_list_limit=24

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 280 300 15.9 300 325
Waiting: 279 298 15.9 297 324
Total: 281 300 15.9 300 325

Result: 297ms

Aero (24 products per page — default):

Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 2 0.1 2 2
Processing: 153 163 9.0 162 183
Waiting: 152 162 9.1 162 182
Total: 154 165 9.0 164 184

Result: 162ms

4. Search “T-shirt”

Magento 5 results:

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 256 294 41.3 286 392
Waiting: 252 289 41.8 281 388
Total: 256 294 41.3 287 392

Result: 281ms. However, the worst result is 388ms. I think to correctly review performance we should use the slowest result. However, it is in the next test. In this test, we will use an average time in favor of Magento 2.

Aero 7 results:

Aero Results is more nice has layered navigation vs Magento 2.4.0. Even ElasticSearch can’t help Magento 2.4.+ to help fast and good test.

Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 2 0.2 2 2
Processing: 141 154 14.0 150 187
Waiting: 140 153 14.0 149 186
Total: 142 156 14.0 151 189

Result: 149ms the worst result is 186ms

5. CMS Page

Magento 2:

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 160 162 1.5 162 165
Waiting: 159 161 1.1 161 162
Total: 161 162 1.5 162 166

Result: 151ms

Aero:

Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 2 0.0 2 2
Processing: 14 14 0.8 14 16
Waiting: 13 14 0.8 14 16
Total: 15 16 0.8 15 17

Result: 14ms

Cms page is the lightest possible eCommerce Functionality functionality without any heavy Product/CAtegory collection models. . It is something like “Hello World” from the database. ‘Hello World’ Benchmark — mini benchmark, comparing the performance HTTP ‘hello world’ on different freamworks. We can see that Magento 2 stack is 10 times slower than modern php app Laravel stack.

Laravel is a great example of a PHP ecosystem that is innovating and thriving in an incredible pace. Hyva(Magento Frontend)

Magento 2 stack and Laravel(Aero Commerce) stack popularity according google trends:

6. Add to cart AJAX

Magento:

Result: 255ms

Aero:

Result: 55ms

7. Cart Page

Magento:

Result: 450ms

Aero:

Result 51ms

And don't forget that the regular Magento 2 page also needs additional requests to slow backend to fetch additional data(customer, log visitor, dynamic sections, banners, shipping and tax calculation,etc.) to be finished. Just a simple request “/customer/section/load/?sections=compare-products%2Cmessages&force_new_section_timestamp=false&_=1602575313245" takes 150ms. The checkout page needs around 10 slow requests.

8. API Get category Page

Magento 2:

Graph QL get category data by ID

SSH POST Curl request:time curl --request POST   --url https://venia.magento.com/graphql   --header 'cache-control: no-cache'   --header 'content-type: application/json'   --header 'postman-token: 3d74cd1b-eb7f-a191-d115-910f4c0bfa09'   --data '{"operationName":"category","variables":{"currentPage":1,"id":19,"filters":{"category_id":{"eq":"19"}},"onServer":false,"pageSize":6,"sort":{"relevance":"DESC"}},"query":"query category($id: Int!, $pageSize: Int!, $currentPage: Int!, $onServer: Boolean!, $filters: ProductAttributeFilterInput!, $sort: ProductAttributeSortInput) {\n  category(id: $id) {\n    id\n    description\n    name\n    product_count\n    meta_title @include(if: $onServer)\n    meta_keywords @include(if: $onServer)\n    meta_description\n    __typename\n  }\n  products(pageSize: $pageSize, currentPage: $currentPage, filter: $filters, sort: $sort) {\n    items {\n      __typename\n      description {\n        html\n        __typename\n      }\n      id\n      media_gallery_entries {\n        id\n        label\n        position\n        disabled\n        file\n        __typename\n      }\n      meta_title @include(if: $onServer)\n      meta_keyword @include(if: $onServer)\n      meta_description\n      name\n      price {\n        regularPrice {\n          amount {\n            currency\n            value\n            __typename\n          }\n          __typename\n        }\n        __typename\n      }\n      sku\n      small_image {\n        url\n        __typename\n      }\n      url_key\n      ... on ConfigurableProduct {\n        configurable_options {\n          attribute_code\n          attribute_id\n          id\n          label\n          values {\n            default_label\n            label\n            store_label\n            use_default_value\n            value_index\n            swatch_data {\n              ... on ImageSwatchData {\n                thumbnail\n                __typename\n              }\n              value\n              __typename\n            }\n            __typename\n          }\n          __typename\n        }\n        variants {\n          attributes {\n            code\n            value_index\n            __typename\n          }\n          product {\n            id\n            media_gallery_entries {\n              id\n              disabled\n              file\n              label\n              position\n              __typename\n            }\n            sku\n            stock_status\n            __typename\n          }\n          __typename\n        }\n        __typename\n      }\n    }\n    page_info {\n      total_pages\n      __typename\n    }\n    total_count\n    __typename\n  }\n}\n"}'

Result: 337ms

If to test on the same site hosted on Magento Cloud result is

Result: 693ms.

In this Magento 2 test, it is fast because of the proper AWS environment. For other hostings and Magento, Cloud results are x3 slower. Also if to install several Best 3-d party modules regular performance of the Magento pages is not faster than 1000ms.

To check real Magento 2 performance on Magento Cloud read this post:

Aero Commerce:

just simple HTTP request: https://yehor.tqt.io/mens/t-shirts?json

Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 2 0.3 2 3
Processing: 33 38 4.0 36 44
Waiting: 33 38 4.0 36 44
Total: 34 40 4.0 39 46

Result: 36ms

Performance Pivot table and Charts

Stacked column chart:

Magento 2.4.o vs Aero Commerce Performance

A stacked column chart allows part-to-whole comparisons. In a stacked column chart, data series are stacked one on top of the other in vertical columns. Stacked column charts can show performance differences because it’s easy to compare total column lengths. Less is better.

Standard chart:

Magento 2.4 vs Modern eCommerce Performance

Lighthouse PageSpeed Insights Performance Product page:

Magento 2:

source: https://twitter.com/hashtag/memento

Broken Magento's default JS Bundling is disabled to improve performance.

Mobile Performance:

Lab Data:

First Contentful Paint: 2.2 s

First Contentful Paint marks the time at which the first text or image is painted.

Speed Index: 18.1 s

Speed Index shows how quickly the contents of a page are visibly populated.

Largest Contentful Paint: 3.4 s

Largest Contentful Paint marks the time at which the largest text or image is painted.

Time to Interactive: 29.0 s

Time to interactive is the amount of time it takes for the page to become fully interactive.

Total Blocking Time: 200 ms

Sum of all time periods between FCP and Time to Interactive, when task length exceeded 50ms, expressed in milliseconds.

Cumulative Layout Shift: 0.439

Cumulative Layout Shift measures the movement of visible elements within the viewport.

Desktop Performance:

Lab Data:

First Contentful Paint: 0.7 s

Speed Index: 5.5 s

Largest Contentful Paint: 1.2 s

Time to Interactive: 10.0 s

Total Blocking Time: 0 ms

Cumulative Layout Shift: 0.086

Result: 56 Mobile and 68 Desktop

Aero Commerce:

All setting is default

Mobile Performance:

Lab Data:

First Contentful Paint: 1.0 s

Speed Index: 1.8 s

Largest Contentful Paint: 1.7 s

Time to Interactive: 2.5 s

Total Blocking Time: 90 ms

Cumulative Layout Shift: 0

Desktop Performance:

First Contentful Paint: 0.3 s

Speed Index: 0.6 s

Largest Contentful Paint: 0.7 s

Time to Interactive: 0.7 s

Total Blocking Time: 20 ms

Cumulative Layout Shift: 0

Aero Commerce frontend performance is awesome. Magento even can’t pass Web Vital requirements.

Web Vitals - quality signals key to delivering great UX on the web (https://web.dev/vitals). This extension measures the Core Web Vitals, providing instant feedback on loading, interactivity, and layout shift metrics.

Magento 2.4.o and Aero Commerce Performance Pivot table:

Page Speed Magento vs Aero (more is better)

Content Breakdown Magento 2 Product Page:

Aero Commerce Content Breakdown Product Page:

My suggestions to improve Aero Commerce performance:

I would like to suggest for the Aero commerce teem to improve Mobile blocking time performance by fixing these several issues:

<script>!(function(d){var b=d.getElementById(‘body-loading’);b.parentNode.removeChild(b)})(document);!(function(w){w.toggleClass=function(s,l){if(s.classList)s.classList.toggle(l);else{var a=s.className.split(“ “),e=a.indexOf(l);e>=0?a.splice(e,1):a.push(l),s.className=a.join(“ “)}};w.addClass=function(s,a){s.classList?s.classList.add(a):s.className+=” “+a};w.removeClass=function(s,e){s.classList?s.classList.remove(e):s.className=s.className.replace(new RegExp(“(^|\\b)”+e.split(“ “).join(“|”)+”(\\b|$)”,”gi”),” “)};})(window);</script>

Style Recalculation on mobile is really slow.

and next issue timer Fired event is to slow on mobile

and

client.js compilation is too slow. It is better to divide it into 3 chanks to avoid main thread blocking for a long time. Code splitting is a great way to improve its initial loading speed. Since a user doesn’t have to download all the code in one hit, they’ll be able to see and interact with the page sooner. This file loading VueJS however where it used on this page I hadn’t found. Maybe it is a redundant dependency on this page.

With Magento everything clear: it is to broken to be fixed. Even Mage Puck couldn't fix its bad frontend performance.

Magento 2 vs Shopware 6 peromance:

--

--

Yegor Shytikov
The Startup

True Stories about Magento 2. Melting down metal server infrastructure into cloud solutions.