Magento Commerce— Staging — What It Is and Its Dark Secret
Magento2 Commerce has introduced a while ago a “staging” functionality. This functionality, which on paper is great has a dark secret.
First of all, let us see what is this staging functionality?
You might need a certain page, such as a CMS page, or a product, or even a category page to change at a very specific time & date. For example, you are selling a product for Christmas, and you would like to stop selling it on the 24th at midnight.
The staging functionality allows you to configure in the back office multiple states for a product, a category, or a CMS page. We can, for example, create a stage for our product in order for it to be enabled on the 23rd of December and disabled on the 24th at midnight. This means customers visiting the website on the 22nd at 23h59m will not be able to find the products. But if they visit the website on the 23rd at 00h01m they will (nearly).
You can use this to change any product attributes; you can also use it to change a category page to add temporarily a banner for example on the page, or to publish the category at a given time.
Without this, you would need to have an admin to the changes at a given time, and even then he might be late doing some of the changes; make a mistake… This automates it all.
You can go quite far and have consecutive stages; again if we keep our Christmas theme, we could make an advent calendar like the display on a CMS page. Every day the page would automatically update. You can even see a preview of the page at a given stage which allows you to validate your changes.
There are of course a few limitations to this feature, one of them is timezones. If your website is selling on multiple timezones and you wish your changes to take effect on the timezone in question you won’t be able to do it. There are a few ways around this but it requires a lot of duplications and still has limitations.
The second issue comes from the way the data is stored. So let us see how the staging works in order to understand the issue.
How are the stages stored?
Magento uses EAV to store its values, so the name of a product for a certain store will be stored as follows:
This basically means product “4” has one entry in the catalog_product_entity table and multiple entries in the catalog_product_entity_varchar table to store all the text values.
When the staging modules is enabled the structure changes,
The catalog_product_entity table changes quite a lot; we have 3 new columns added to the table. entity_id is no longer the primary key of the table, now it’s row_id. This means there can be multiple rows for the same product. The created_in & updated_in columns allow us to load the “proper” stage at a given time.
Let us also note the values store in the varchar table has also been duplicated for each row. So instead of having one value per product, we have one value per product & version. Which seems logical.
With staging, all queries to products are automatically added a new condition :
1615385079 BETWEEN main_table.created_in AND main_table.updated_in
With this, all queries using the product collection will load a product only once even if it has multiple stages. How come? If we have a stage that starts the 24th and ends the 25th the base stage that starts at 1 and never ends the query should find both?
Yes, that is why the “main” stage is split into 2 in the database:
So, we have 3 duplicates of the product stored in the database. This allows the stages to work quite efficiently on the frontend. But how does edition works?
Let’s say we created our stage “Christmas stage” and we are still on the 21 of December. We now fix something on our product; the changes are made to the first stage that starts at 0 and ends the 24th? And when the 25th arrives our changes disappear? No there is an additional table, staging_update which links these “periods” in the database between each other. So when you edit the “Main Stage” it will in the database update entries for the 0–24th period and also the 25th to infinity period.
You probably are starting to see the problem here.
Why is this a problem?
The problem arises when you start having a lot of stages:
We have 5 stages, yet in the database, the data is duplicated 9 times. That means that ((2 * nb_stage) -1) entries in the database. So we are duplicating data in the database which we know is a bad practice and I don’t think needs time explaining here.
The second issue is more practical; it arises when we try to save the product. The more stages you have the more the “Main Stage” becomes fragmented and hard to update; making the save operation on a product slower and slower.
Finally, there are not cleanup methods, once easter is passed we could imagine deleting the old stages and remove some of the duplicated data. There is nothing doing this natively.
The staging modules of Magento can be really useful, more so on less complex objects such as CMS pages or price rules. If your database is relatively small it can also safely be used on products and categories.
My main issue with it is that it’s installed by default in the Commerce Edition of Magento it should have been optional for integrators to install. The module modifies the database structure extensively, which means it can not be easily removed once installed. So better install it only when needed. A lot of projects has the modules installed and enabled without reason, often developers working on the website don’t take it into account which results in errors when it is actually used. So it’s prone to create bugs.