Native Magento Data Indexer

Piotr Karwatka
Jun 19 · 2 min read

Vue Storefront is a platform-agnostic solution. Our standard data importers were designed to solely use the platform APIs to push the data into Elastic Search. This is how the mage2vuestorefront works.

As flexible, this solution is, it comes with some serious limitations. One of the most important limitations it has is the performance. Executing API requests for each product might have a sense for 1–2K SKUs but doesn’t work with the databases counting a hundred thousand items.

The most daring, in this case, was the large product catalog of around 1 Million SKUs — an amount for which mage2vuestorefront would roughly take 8 days while totally killing the server. Fortunately, magento2-vsbridge-indexer came just in the right moment and does the same thing in 19 minutes — Michael Eissmann, Klebefieber

klebefieber.de is one of the first production deployments of native indexers — with the database of 500,000 SKUs (full re-index: 19minutes)

This was our motivation to create the native indexers for Magento. Currently, you can find two native indexers on our Github:

Some cool features:

  • both are native Magento indexers with support for on-demand indexing,
  • custom URL (1.9)+ MSI support (1.10rc-1),
  • configuration panel,
  • PHP based :-),
  • performance — tested with the SKU databases of 500,000+,
  • supports VS SSR cache invalidation.

The native indexer updates the Elastic Search index in the very same format as the mage2vuestorefront. Our intention was to speed up the indexation process and make it more reliable. With native indexer, we were able to use the Magento2 ORM and events to optimize the indexation process. Please do use this module instead of mage2vuestorefront if you experience any issues regarding indexing performance. Both projects are currently officially supported by the Vue Storefront Core team.

The indexers were brought to life by our own Agata Firlejczyk and now maintained by the community. Thank you, Agata — great job :)

Learn more about Vue Storefront.

The Vue Storefront Journal

The official journal of Vue Storefront community. The latest news on meet-ups, contributions, releases, and the community surrounding Vue Storefront — the top open source solution for progressive web apps dedicated to mobile-first eCommerce. (www.vuestorefront.io)

Piotr Karwatka

Written by

The Vue Storefront Journal

The official journal of Vue Storefront community. The latest news on meet-ups, contributions, releases, and the community surrounding Vue Storefront — the top open source solution for progressive web apps dedicated to mobile-first eCommerce. (www.vuestorefront.io)