Web Data & E-commerce Monitoring: Beyond The Price Monitoring Paradigm

When you talk about Web Data and E-commerce, the first thing that comes in stakeholders’ mind, is price monitoring. Pricing is still a key need for retailers. That’s why several Data-providers are really focused on pricing. However, building (or using) a Web-scale (Big Data) E-commerce platform can disrupt the landscape of a two figures growing business. Here’s why.

Three ways to disrupt retail by a Web-Scale supply monitoring

The LIRIMA (International Laboratory for Computer Sciences and Applied Mathematics), backed to INRIA, organized a Workshop about Data Sciences yesterday in ENIT — Tunis. Houda Bakir and I participated to the closing panel. Here is a sneak peek of our contribution. We are trying to reword and to share scientific issues from a business point of view, and vice versa.
LIRIMA Workshop on Data Sciences 2017. Credit — Prof. Mohamed Jaoua.

Technology that can do discovery for You

If you handle hundreds of SKUs in your SOHO, and if you run your business in a highly competitive area, you can easily find dozens of tracking solutions tailored to your needs for a few dollars per month. Prisync, Sniffie or Paarly can be really amazing for you.

Usually you have to bootstrap them by setting the products lists (some URLs to track), but it is not a big deal : the Excel file you use to do your manual tracking is generally enough.

However, if you are a mid-size retailer, or a top player, you have to upgrade your monitoring to fit at least the top of the mind’s competitors sites. We talk about twenty E-Shops of thousands products each one, it can quickly increase to 100,000 URLs, 1,000,000 URLs, couple of millions URLs. You may fill your Excel file, but you will have to deal with your daily headache.

Few providers in the market offer solutions that can scale to hundreds of Web sites with no need to provide your Excel list of URLs. Not only they can crawl and extract daily exhaustive data from all your competitors shops, but they can also perform the discovery for you: every single new product is automatically monitored whenever it is available online on your competitors shops. They give you information about new products and about competitors that you didn’t know the day before. Thus, it will change your life.

The moment you can grab the full E-shops supply, a new set of amazing features become within reach: price distribution, policies analytics, brands exposure, distribution channels, etc. You get the full view and a 360° market view.

A breakthrough product matching that works

When you first start competitor’s price monitoring, you are very happy by crunching data every day. But enthusiasm fades away quickly when you have more specific needs. Comparing your price to competitors assumes that you are handling the same products. This is called product matching.

Again, if you deal with a few hundreds product references, your Excel file still works. Solutions that require manual product matching still work.

But if you manage an assortment around ten thousands of SKUs, you have to monitor ten or twenty times more urls from your competitors shops. So, even if you are focused on 15% of your assortment, for instance your flagships and your best-sellers, you can no longer make your product matching manually, reference by reference, URL by URL.

Alternatives like low cost contractors are not enough, and can’t scale anymore. Naive solutions like using EANs, SKUs to match products do not work: who publishes EANs within the product details anyway?

Several E-commerce monitoring providers innovate with data science. For instance, advanced text analytics do very good product matching.

An amazing solution may rise from an exclusive combination of natural language processing and computer vision. The product details are a datamine: product name, technical specifications, brand, free text description and pictures.

Imagine a powerful AI with billions of such data-points: That’s Datavora. This breakthrough technology changes the way you monitor the market. It matches products as good as an expert. Furthermore, this AI can scale to millions of URLs.

Beyond High Frequency Pricing : Predicting Prices

Try to take a photo of a formula one race from the paddock with your smartphone. The picture may be blurry. Cars go so fast that things change while you are taking that pic.

Amazon changes the price of few percents of the assortment more then twenty times each single day. You may be building your homemade Web data collection. You may have several contractors and data providers. If you are competing in a slow moving trade, it’s fair enough, but if you compete with F1, you need to switch to the next level. Smartphone pics are useless.

Few years ago, the industry standard was an exhaustive weekly update. The trend today is to grab data daily, several times a day.

Prisync, Netveille and Pricesquid are updating data few times a day. Datavora does the same. The best solution for your business depends mainly on data volume. We built a scalable and unique platform that can daily grab millions of updates from of hundreds E-shops and marketplaces.

Few months after this achievement, we cumulated amazing and never seen before timeseries of prices over 1800 E-Commerce Web sites. Billions of datapoints are already stored. We can start predicting prices.

You already have the extraordinary tool that gives your competitors’ price with few hours delay? Imagine you have the oracle that can tell you whether to lower or increase your prices tomorrow.

If you need an amazing camera to make beautiful pics of a formula one race, bring Datavora to the paddock. You’ll be able to optimize your supply and adjust your prices in few minutes, even before the effects of competitor promotions are reflected on your sales.

If you are familiar with the trading rooms and stock markets, this is like high frequency trading: you will be able to take action before the competitor decision effects. For you, it’s a big milestone!

Stay tuned to get more details. You have a frustration with your competition monitoring or Web data? We can start discussion right now. We could meet soon, on our lovely road trip to Paris, Madrid and Lisbon this autumn, or we can get in touch through.