Flipkart Lite — The how?

8 min readMar 18, 2016

To know what Flipkart Lite is, read this article by Aditya Punjani on the story behind building the Progressive Web App,

Building Flipkart Lite: A Progressive Web App

There have been a few turning points in the history of the web platform that radically changed how web apps were built…

medium.com

Also, this article is cross posted here at the Official Flipkart Tech Blog — http://tech-blog.flipkart.net/2016/03/flipkartlite-the-how/

Well, where do I even start? The following is the list of most of the tech behind Flipkart Lite in NO particular order. Many thanks to all the authors and contributors of these tools, libraries, frameworks, specification etc…

Front-end:

Tools — Build and Serve:

Web platform:

And a few more that I’m probably missing.

Some of the things listed above come with the tag “Experimental Technology”. But, trust me, take a leap of faith, play around, implement it in your app and push it to production. You’ll definitely feel proud.

The Monolith

The architecture decision started with choosing between “One big monolith” and “Split the app into Multiple Projects”. In our previous projects we had one big monolith repository and the experience getting one’s code to master wasn’t pleasant for many of the developers. The problem having all developers contributing to one single repository is — facilitating them to push their code to production quickly. New features flow in every day and the deployments kept slowing down. Every team wants to get its feature into master and effectively into production ASAP. The quintessential requirement of every single person pushing his/her code to his/her branch is that it should get reviewed and merged in the next hour, pushed to a pre-production like environment in the next 5 minutes and ultimately production on the same day. But how can you build such a complicated app from scratch right from the day 1.

We started with a simple system and added complexity

Separate concerns

The goal was to have the following DX(Developer Experience) properties,

To develop a feature for a part of the application, the developer needs to checkout only the code for that particular application
One can develop and test one’s changes locally without any dependencies on other apps
Once the changes are merged to master, these changes should be deployed to production without depending/waiting on other applications

So we planned to separate our app into different projects with each app representing a product flow — ex: Home-Browse-Product or the pre-checkout app, Checkout app, Accounts app, etc… but also not sacrificing on some of the common things that they can share with each other. This felt like restricting usage in technology. But everyone agreed to use React, Phrontend and webpack :). So it became easy to add simple scripts into each of the project and automate the build and deploy cycles.

Out of these different apps, one of them is the primary one — the Home-Browse-Product app. Since the user enters Flipkart Lite through one of the pages in this app, let’s call it the “main” app. I’ll be talking only about the main app in this article as the other apps use a subset of tooling and configuration the same as in the “main” app.

Home-Browse-Product

The “main” app takes care of 5 different entities. Since we use webpack, each of the entities just became a webpack configuration.

vendors.config.js : react, react-router, phrontend, etc…
client.config.js : client.js → routes.js → root component
server.config.js : server.js → routes.js → root component
hbs.config.js : hbs-loader → hbs bundle → <shell>.hbs
sw.config.js : sw.js + build props from client and vendor

vendors.config.js

This creates a bundle of React, react-router, phrontend and a few other utilities.

{
  entry: {
    react: "react",
    "react-dom": "react-dom",
    "react-router": "react-router",
    phrontend: "phrontend",
    ...
  },
  output: {
    library: "[name]",
    libraryTarget: "this",
    ...
  },
  plugins: [
    new webpack.optimize.CommonsChunkPlugin({ name: "react" }),
    new CombineChunksPlugin({ prelude: "react" })
  ]
}

This is roughly the configuration for the vendors js. We tried different approaches and this worked for us. So the concept is to

specify different entry points
use CommonsChunkPlugin to move the runtime to this chunk
use CombineChunksPlugin (I’ve uploaded the source here) to combine all the chunks to one single file with the runtime chunk at the top

As a result, we get one single file — vendors.<hash>.js, which we then sync to our CDN servers.

The vendors list is passed down the build pipeline to other apps as well and each of them (including main) externalise the vendors so as to be used from vendors.bundle.js,

{
  externals: vendors
}

client.config.js

As the name says, it is the configuration to build the app. Here we do minification and extraction of css into a single file.

server.config.js

Why do we even need a server.config.js ? Why do we need to bundle something for server?

We use webpack heavily and we rely on some conventions — we add custom paths to module resolution, import css files, use ES6 code, and node cannot understand most of this natively. Also, we don’t want to bundle every single dependency into one single file and run it with node. Webpack provides a way to externalise those dependencies leaving the requires for those externals untouched and this is one way of doing it,

var nodeModules = {};
fs.readdirSync(‘node_modules’).filter(function(m) {
  return [‘.bin’].indexOf(m) === -1;
}).forEach(function(m) {
  nodeModules[m] = ‘commonjs2 ‘ + m;
});

and in the webpack configuration,

{
  output: {
    libraryTarget: "commonjs2",
    ...
  },
  externals: nodeModules,
  target: "node"
}

sw.config.js

The sw.config.js bundles sw-toolbox, our service-worker code and the Build versions from vendors.config.js and client.config.js. When a new app version is released, since we use [hash]es in the file names, the URL to the resource changes and it reflects as a new sw.bundle.js. Since we now have a byte diff in the sw.bundle.js, the new service worker would kick in on the client and update the app and this will work from the next Refresh.

hbs.config.js

We use two levels of templating before sending some markup to the user. The first one is rendered during the build time hbs → hbs. The second one is rendered during runtime hbs → html. Before proceeding further into this, I’d like to talk about HTML Page Shells and how we built them with react and react-router.

HTML Page Shells

Detailed notes on what HTML Page shells or Application Shells are are given here — https://developers.google.com/web/updates/2015/11/app-shell . This is how one of our page shells look like —

The left image shows the “pre-data” state — the shell, and the right one is the “post-data” state — the shell + content. One of the main things this helps us in achieving is this —

Perceived > Actual

It’s been a thing for quite some time that what the user perceives is the most important in UX. For example, splashscreen — it informs that something is loading and gives the user some kind of progress. Displaying a blank screen is no use to the user at all.

So I want to generate some app shells. I have a react application and I’m using react-router. How do I get the shells ?

Gotchas ? maybe.

We did try some stuff and I’m going to share what worked for us.

componentDidMount

This is a lifecycle hook provided by React that runs ONLY on the Client. So on the server, the render method for the component is called but componentDidMount is NOT invoked. So, we place all our API calling Flux actions inside this and construct our render methods for all our top level components carefully such that once it renders, it gives out the Shell instead of throwing an empty container.

Parameterised Routes

We decided that we would create shells for every path and not a simple generic one that you can use for anything. We found all the paths that the application used and would use. We wrote a small script that iterated through all the routes we defined to react-router and we had this —

/:slug/p/:itemid
/(.*)/pr
/search
/accounts/(.*)

During build time, how can you navigate to a route (so as to generate HTML Page Shells for that route), with an expression in the route that is resolved only during run time ?

Hackity Hack Hack Hack

React-router provides a utility function that allows you to inject values to the params in the URI. So it was easy for us to simply hack it and just inject all the possible params in the URIs.

function convertParams(p) {
  return PathUtils.injectParams(p, {
    splat: 'splat',
    slug: 'slug',
    itemId: 'itemId'
  });
}

Note: We used react-router 0.13. The APIs might have changed in later versions. But something similar would be used by react-router.

And now we get this route table.

Route Defined     → Route To Render    → HTML Page Shell
/:slug/p/:itemid  → /slug/p/itemId     → product
/(.*)/pr          → /splat/pr          → browse
/search           → /search            → search
/accounts/(.*)    → /accounts/splat    → accounts

It simply works because the shell we are generating does NOT contain any content. It is the same for all similar pages (say product page).

Two-level templating

We are back to hbs.config.js. So we have one single hbs file — index.js with the following content.

// this is where the build time renderedApp shell goes in
// React.renderToString output
<div id="root">{{content}}</div>// and some stuff for second level
// notice the escape
<script nonce="\{{nonce}}">
<script src="\{{vendorjs}}"></script>
<script src="\{{appjs}}"></script>

Build time rendering: For each type of route, we generate a shell with $content injected into index.hbs and get the hbs for that particular route, example — product.hbs

Runtime rendering: We insert all the nonce, bundle versions and other small variables into the corresponding app shell hbs and render it to the client.

The good thing is that the user gets a rendered html content before static resources load, parse and execute. And this will be faster than server-side rendering, as this is as good as serving a static HTML file. The only variables in the template are a few numbers that are fetched by in-memory access. This improves the response-time and the time to first paint.

The end

And with all this and a lot of gotchas that I missed out, we put Flipkart Lite into production successfully. Oh wait! This didn’t start as a story. Anyway. Thanks for reading till here.

…

All the solutions described above are NOT necessarily the best solutions out there. It was one of our first attempts at solving them. It did work for us and I’m happy to share it with you. If you find improvements, please do share it :).