The internet isn’t as transparent as you think

How the structure of a human-readable web creates barriers for e-commerce price comparison

In theory, e-commerce shopping promises a lot: it offers the holy grail of an efficient marketplace, giving you the best prices, allowing you to view, compare, and purchase from a trillion-page catalog, all from the comfort of your home. A lot of the legwork is being done for you; efficient logistics (with home delivery), 360-degree pictures, an army of apps that offer beautifully designed shop windows for you to look.

But what about pricing? Not as easy as you think.

Although the internet offers the ability to compare various prices between stores, it has a limit: webpages are designed to be read by humans.

And that’s a problem.

In order to do a price comparison, you’d open a window on one store, open another window on another store, look for similar products and compare.

Now imagine doing that for millions ofproducts. Imagine an army of interns manually inputting price points for specific UPCs across various domains. Its soul-crushing stuff.

The thing is, retailers want you to do just that. They want you to give up. They want you to be sold on a fancy site presentation, and not get the best value on a deal. Because when you give up looking for cheaper prices, you’re forced to take what they give you.

Increasingly online retailers have been trying to hinder price comparison on their websites. More and more retailers are hiding their prices behind login pages, or employing a “price-in-cart” strategy, where users are not allowed to see a price until they add the product into their shopping cart. This is making price transparency increasingly difficult

The architecture of the internet also makes it incredibly hard to get instant visibility on the marketplace of online goods since its a human-readable web. And that’s a problem. Because when you can’t easily compare goods and prices, you don’t have an efficient market. You open yourself up to monopolistic manipulation.

An efficient marketplace helps everyone by allowing the true value of a good shine through. Complete visibility and transparency.

The online retail industry is ready for a transformation similar to the airline industry. Airline ticket comparison websites like kayak.com and skyscanner.com helped break down the airlines’ key weapon: making it difficult to compare ticket prices. It forced airlines to commoditize air travel, making it cheaper for everyone.

To make a good price comparison website, you need to be able to get a computer to read the web like a human. That’s where e-commerce price and product data extraction come in.

I help run a little startup called Semantics3 where we’re trying to accomplish for e-commerce retail, what Kayak, Skyscanner and the whole host of travel sites did for the airline industry.

We want to build a more efficient marketplace, by increasing pricing transparency across online retailers. We did this by creating a online database of e-commerce product and pricing data that allow you to compare a product through a URL on a specific retailer, against other retailers.

By creating an API that anyone can access, you can get a direct product feed that’s structured in a format that easily read by your program (JSON)

This means you can programmatically buy stuff online, making sure you get the best price always!

Cool beans, innit?