How to use Webpack’s new “magic comment” feature with React Universal Component + SSR

Webpack 2.4.0, which came out a few weeks ago, launched with a very interesting new feature: “magic comments.” In combination with dynamic imports, “magic comments” greatly simplify code-splitting + server-side rendering. Now you can name the chunks corresponding to the modules/components you import. Prior to this an enormous amount of work was needed to determine what chunks to serve in initial requests so async components could also render synchronously.

Here’s all it looks like:

() => import(/* webpackChunkName: 'Anything' */ './MyComponent')

Recently I wrote about how to use webpack-flush-chunks to cross-reference moduleIds flushed from react-universal-component or react-loadable with Webpack stats to generate the precise scripts and stylesheets to send from the server. However, naming chunks greatly greatly simplifies this process. webpack-flush-chunks supports this as well, but it’s very important to know how to do this manually since inevitably you will have custom needs.

We’ll start with the server and work our way backwards, saving the magic for last.

STEP 1 — render + flush chunks

To begin we’re going to assume you have a method to flush the chunk names rendered on the server. For now we’ll assume you can easily do this.

const appString = ReactDOMServer.renderToString(<App />)                                 
const chunkNames = flushChunkNames()

Now, you should have an array of chunk names that look like this:

[‘chunk1’, ‘chunkWhatever’, ‘my-chunk-name’, ‘etc’]

STEP 2 — get the files from stats

We’re going to skip a detailed explanation of how to get Webpack compilation stats.

In production, I recommend you use the Webpack plugin, stats-webpack-plugin, and during development I recommend you use webpack-hot-server-middleware to usher the stats to your serverRender function. You can use one of my boilerplates to give all this a try in a few commands: https://github.com/faceyspacey/flush-chunks-boilerplate-webpack-chunknames

Once you have the stats, you need to know where to find your chunks and the files they contain. They exist within the stats.assetsByChunkName:

stats.assetsByChunkName = {
   chunk1: ['0.js', '0.js.map', '0.css', '0.css.map'],
   chunk2: ['1.js', '1.js.map', '1.css', '1.css.map'],
   chunk3: ['2.js', '2.js.map', '2.css', '2.css.map'],
   main: ['main.js', 'main.js.map', 'main.css', 'main.css.map'],
   vendor: ['vendor.js', 'vendor.js.map'],
   bootstrap: ['bootstrap.js', 'bootstrap.js.map']
}

Having familiarity with what assetsByChunkName looks like is half the battle. Notice a “chunk” isn’t just a javascript file, but an array of files.

Because you want an array rather than array of arrays, the primary thing we do here is pick the chunks we’re interested in (which are arrays), and then combine/flatten them into a single array:

const assets = webpackStats.assetsByChunkName
const filesForChunk = chunk => assets[chunk]
const files = flatten(chunkNames.map(filesForChunk))
function flatten(array) { 
   return [].concat(...array)
}

STEP 3 — filter your scripts and stylesheets

This is the easiest part. If things had been this frictionless all along, code-splitting in combination with SSR would have been a far easier nut to crack:

const scripts = files.filter(f => f.endsWith('.js'))
const stylesheets = files.filter(f => f.endsWith('.css'))
// and for good measure let's get the publicPath before proceeding
const path = webpackStats.publicPath

STEP 4 — put it all together and serve it

Now it’s just a matter of creating strings (or perhaps React elements for ReactDOMServer.renderToStaticMarkup) to send to clients:

export default function serverRender(req, res) {
  const appString = ReactDOMServer.renderToString(<App />)                                 
  const chunkNames = flushChunkNames() // will get to this soon
  const assets = webpackStats.assetsByChunkName
  const filesForChunk = chunk => assets[chunk]
  const files = flatten(chunkNames.map(filesForChunk))
  
  const scripts = files.filter(f => f.endsWith('.js'))
  const stylesheets = files.filter(f => f.endsWith('.css'))
  const path = webpackStats.publicPath
  
  res.send(
    `<!doctype html>
      <html>
        <head>
          ${stylesheets
              .map(f => `<link href='${path}/${f}' />`)
              .join('\n')
          }
        </head>
        <body>
          <div id="root">${appString}</div>
          ${scripts
              .map(f => `<script src='${path}/${f}'></script>`)
              .join('\n')
          }
        </body>
      </html>`
  )
}
            
const flatten = (array) => [].concat(...array)

In a less naive example you would insure that your bootstrap.js script comes first, followed by chunks, and ending with main.js. It’s a chore to achieve this. This is one of the things webpack-flush-chunks automatically does for you.

If you were looking for a quick overview of how to weed your way through Webpack stats to utilize the new “magic comment” feature, hopefully your pleasantly surprised with how easy it is.

What’s left is:

• actually telling Webpack to generate these chunks
• demarcating in code when they are in fact used
• and then flushing the ones used.

PART 2: USING MAGIC COMMENTS TO CREATE THESE CHUNKS

STEP 1 — create <UniversalComponent />

To put Webpack chunk names to use you need a component or package that lets you mark what is considered a chunk and when it is used. Calling import with a magic comment is the easy part. You need your React component to also register the usage of a chunk by the same name. react-universal-component does this for you via the chunkName option:

import React from 'react'
import universal from 'react-universal-component'
const asyncComponent = 
   () => import(/* webpackChunkName: 'Anything' */ './MyComponent')
const UniversalComponent = universal(asyncComponent, {
  resolve: () => require.resolveWeak('./Foo'),
  chunkName: 'Anything'
})
export default () =>
  <div>
    <UniversalComponent />
  </div>

The important part is that the string you provide for webpackChunkName must match the chunkName option. Notice they are both ‘Anything’. It’s a bit redundant, but the former is a static comment and there’s not much we can do about that. From my perspective, expending these lines is the least of my worries. I’m very thankful this feature is finally in Webpack.

What is more nuanced though is when asyncComponent is called. You very well could provide a standalone promise rather than a function. Eg: import(.. without the arrow function part. But then the client would make an additional request to get that component immediately on page load, even if you did not render it. By guarding the promise with a function, you guarantee it’s not called until <UniversalComponent /> is rendered. React Universal Component handles that internally for you.

But what’s the resolve option?

This ultimately is the ultimate trick here. It gives your Webpack server a synchronous way to require your component, without the client including it in the parent chunk’s dependencies. See, if you did this: resolve: require('./Foo'), just by the existence of that in main, Webpack would include the very code you are trying to move into another chunk. It would defeat the purpose of code-splitting.

In addition, require.resolveWeak is used by the client on page load when your app is rendered for the first time and you have correctly provided its corresponding chunk (as per Part 1). This avoids React “checksum mismatches” which lead to an additional render. And more importantly this prevents that unnecessary second request for a chunk corresponding to what was already rendered correctly by SSR.

See, even if you correctly rendered the component you want split on the server via renderToString and sent that to the client, the client would soon after replace it with your <Loading /> component that main.js expects to render, while it fetches its chunk anyway. Not good.

But does Babel have require.resolveWeak?

If you are using Babel for your server, you must also provide the absolute module path:

const UniversalComponent = universal(asyncComponent, {
  resolve: () => require.resolveWeak('./Foo'),
  chunkName: 'Anything',
  path: path.join(__dirname, './Foo')
})

The rest is left up to React Universal Component to dynamically toggle between using one of the 3 methods of importing your module depending on the environment.

The universal HoC takes several more options (all of which are optional). To learn about them, visit:

faceyspacey/react-universal-component

Here’s a quick summary of the options:

• error — optional <Error /> component
• loading — optional <Loading /> component
• key — used to find the export you want in a given module
• onLoad — used to take other exports from a module and do things like replace reducers
• minDelay — controlled delay to prevent jank when switching from your loading component to your primary component
• timeout — maximum loading time before error component shows
• chunkName — the name of the chunk that will be flushed if rendered

STEP 2 — FLUSHING THOSE CHUNKS

If you’ve done all correctly up to this point, flushing chunks is just a matter of making an additional require in serverRender and calling it with a few parameters:

import flushChunks from 'webpack-flush-chunks'
import * as Universal from 'react-universal-component/server'
const appString = ReactDOM.renderToString(<App />)
const { js, styles } = flushChunks(webpackStats, {
  chunkNames: Universal.flushChunkNames(),
  before: ['bootstrap', 'vendor'],
  after: ['main']
})
res.send(`
  <html>
    <head>
      ${styles}
    </head>
    <body>
      <div id='root'>${appString}</div>
      ${js}
    </body>
  </html>
`)

The key thing to recognize here is that because of the way Node works (i.e. regarding the event loop) and the way renderToString works (i.e. synchronously), you can perform other synchronous actions immediately after, which make use of some global state (namely the arrays/sets behind the scenes containing your chunkNames) and guarantee that they are reset for queued requests. flushChunkNames() clears the set of chunk names recorded in the render, and as a result no annoying React provider components are necessary. Just make sure not to call await or kick off any promises before calling res.send.

As for the original example in Part 1, obviously you can use Universal.flushChunkNames() in the same way and do the corresponding work to get scripts and stylesheets manually. webpack-flush-chunks also has a Low Level API that can simplify things for the semi-manual route as well. Specifically, it gives you the all files from rendered chunks, without addressing file ordering, creating strings/components, or getting your main/vendor scripts,etc.

The most important thing you will need is the correct Webpack configuration. Make sure to check out that section of the docs.

FAQ

The previous sections were designed to be easy-reading — perhaps if you’re familiar with this stuff, you’re here after just perusing the code examples. The meat is really in the answer to the following questions you may have:

1. What about creating my own HoCs?

Almost everywhere we want to do code-splitting, we want to dynamically do it for a bunch of sections of our site/app at the same time. We want to hit a bunch of birds with the same stone. How can we accomplish the following:

function createUniversalComponent(page) {
  return universal(import(`./pages/`${page}`), {
    resolve: () => require.resolveWeak(`./pages/`${page}`)
  })
}

Your import no longer needs to be a function since it’s within a function not called until “render time.” Magic comments are left out for readability. If you didn’t know, webpack-flush-chunks works without magic comments too. Using moduleIds which you can retrieve from Universal.flushModuleIds(), you can achieve the same on older versions of Webpack!

The way we implement this is a bit different since we won’t know the page you want to render until “render time.” Let’s give it a look:

const MyParentComponent = ({ page }) => {
  const UniversalComponent = createUniversalComponent(page)
  return (
    <div>
      <UniversalComponent />
    </div>
  )
}

Now you can create your own HoC and re-use it perhaps for the entire code-splitting needs of your apps. That’s a lot of bytes you’re saving your clients, and at way cheaper developer price than ever before. And most importantly, SSR is no longer a tradeoff you have to make.

There is one problem with this. Currently require.resolveWeak can’t resolve dynamic imports like import can. I.e. you can’t have a string such as './page/${page}' where part of the path is dynamic. If you don’t already know, Webpack is perfectly capable of doing this. The way it handles it is by creating chunks for every module in the page folder.

I’ve created an Issue for this and Tobias Koppers has recently prioritized it with Important. Vote it up if you want this fixed.

Here’s how you accomplish this for now (and it’s likely similar to what you’ve been doing in the past, just without SSR):

const Page1 = universal(() => import('./pages/Page1'), {
  resolve: () => require.resolveWeak(('./pages/Page1')
})
const Page2 = universal(() => import('./pages/Page2'), {
  resolve: () => require.resolveWeak(('./pages/Page2')
})
const Page3 = universal(() => import('./pages/Page3'), {
  resolve: () => require.resolveWeak(('./pages/Page3')
})
const pages = { Page1, Page2, Page3 }
const createUniversalComponent = page => pages[page]

And the implementation of <MyParentComponent /> is the same as before.

One thing to note about this is its performance characteristics. This actually does less work at “render time,” since the components are pre-created. You’re essentially sacrificing memory for CPU at a very critical point in time. I haven’t measured how many cycles/milliseconds creating these components during every render is, but it’s probably negligible. I’m definitely looking forward to doing the initial HoC implementation. You can also reduce work during render by just creating the component once during lifecycle methods such as componentWillMount and componentWillReceiveProps, and then set them as state. So you have options here.

The main takeaway is you WANT the capability to dynamically choose which “universal component” to render.

In the future we will likely offer this interface as well:

const create = universal(({ page }) => import(`./pages/${page}`), {
  resolve: ({ page }) => require.resolveWeak((`./pages/${page}`),
  path: ({ page }) => path.join(__dirname, `./pages/${page}`)
})
const UniversalComponent = create()
<UniversalComponent page={page} />

Then you don’t even need to create your own HoCs, and you avoid wasting precious CPU cycles at render time. Best of all worlds.

2. What about preloading my components?

Often times your users have few options of where to navigate to and you want to preload all options to optimize experience. You can do so like this:

import universal from 'react-universal-component'

const UniversalComponent = universal(() =>  import('../components/Foo'), {
  resolve: () => require.resolveWeak('./Foo')
})

export default class MyComponent extends React.Component {
  componentWillMount() {
    UniversalComponent.preload()
  }

  render() {
    return <div>{this.props.visible && <UniversalComponent />}</div>
  }
}

3. Can I trigger the loading component while I fetch data?

What’s been covered very little in this article is the <Loading /> and <Error /> components that React Universal Component displays for you as needed. It’s an obvious capability, but what isn’t is that you too can trigger them, say, if you’re fetching data in a parent component. It’s very intuitive and saves you from repeating yourself. Here’s how you do it, for example, using Apollo’s asynchronous HoCs:

const UniversalUser = universal(() => import('./User'), {
  resolve: () => require.resolveWeak('./User'),
  loading: <Loading />
})
const User = ({ loading, error, user }) =>
  <div>
    <UniversalUser isLoading={loading} error={error} user={user} />
  </div>

export default graphql(gql`
  query CurrentUser {
    user {
      id
      name
    }
  }
`, {
  props: ({ ownProps, data: { loading, error, user } }) => ({
    loading,
    error,
    user,
  }),
})(User)

What this accomplishes beyond just code re-use is less jank in terms of switching between your <Loading /> component and your primary universal component. <UniversalUser /> will show <Loading /> until both the async import resolves AND the data is returned from your GraphQL server (which both can operate in parallel). If you were going the route of 2 separate spinners — it saves you from that trap as well.

As for the server, you will of course use Apollo’s amazing recursive promise resolution solution to populate your component tree with data while the requires from <UniversalUser /> resolve synchronously and in a split second. So it combines nicely in both environments.

4. What about creating CSS chunks?

Anyone who’s successfully done code-splitting (with or without SSR) know that it’s just for your javascript chunks — well, here’s the new hotness:

const ExtractCssChunks = require("extract-css-chunks-webpack-plugin")
module.exports = {
  module: {
    rules: [
      {
        test: /\.css$/,
        use: ExtractCssChunks.extract({
          use: {
            loader: 'css-loader',
            options: {
              modules: true,
              localIdentName: '[name]__[local]--[hash:base64:5]'
            }
          }
        })
      }
    ]
  },
  plugins: [
    new ExtractCssChunks,
  ]
}

Look familiar? It took a lot of work to re-orient this plugin. extract-css-chunks-webpack-plugin also supports HMR for your css — something the original extract-text-webpack-plugin did not. There’s a lot more to this as well, given you have to configure a matching server environment either with Babel or Webpack, create chunks that inject js and chunks that have it removed since stylesheets already have it, etc. More coming about this soon…

CONCLUSION

There are still a few remaining pieces to the puzzle missing. Namely: the ability to generically fetch data along with [multiple] imports in your asyncComponent function. See, it shouldn’t have to only contain a call to import(). You should be able to specify any data dependencies in this promise, perform calculations, even call import() multiple times. I think that sort of frictionlessness is what will evolve the platform.

This capability serves the same purpose as Next.js’ getInitialProps:

const asyncWork = async props => {
  const prom = await Promise.all([
    import('./User'),
    import('./Post'),
    fetch(`/user?id=${props.id}`)
  ])
  const User = prom[0].default
  const Post = prom[1].default
  const data = await prom[2].json()
  return (
    <User data={data} {...props}>
      <Post />
    </User>
  )
}
const UniversalComponent = () => universal(asyncWork)
<UniversalComponent id={123} />

Notice how we’re requesting 2 chunks + fetching some data in parallel!

Being able to code your components without friction like that, and without having to worry about tradeoffs regarding SSR vs. Code-Splitting is the future. If you’ve read Bret Victor’s Ladder of Abstraction article, you know the importance of things becoming frictionless for platforms to evolve. Up until recently we’ve kinda been stalled at being able to get the full use of our platform. There shouldn’t be certain places where we can do the above things and other places we can’t. We shouldn’t be constrained even to the current react-universal-component HoC. There have been solutions that accomplish asyncWork above, but they are async-only. You trade SSR for async-only splitting. That absolutely shouldn’t be the case.

That said, react-universal-component as it is will remain very important, even if/when the aforementioned problem is solved. It will always have a leg up over solutions that recurse your component tree resolving promises in the simple fact that it doesn’t need to waste cycles on your server doing a “pre-render” to find those promises. However, next week I’ll be releasing the best of all worlds, and you can decide based on your needs. Everything covered here will stay the same and I, myself, will 100% continue to use exactly what you’ve seen today. But for developers who like to fetch data in componentWillMount, I have some very exciting things coming your way. Stay tuned!

BOILERPLATES:

https://github.com/faceyspacey/webpack-flush-chunks/#boilerplates

Checkout the 4 boilerplates to frictionless try things out with either a Webpack or Babel server, and with or without “magic comments.”

> For more idiomatic javascript in Reactlandia, read:

Server-Render like a Pro /w Redux-First Router in 10 steps

Redux-First Router data-fetching: solving the 80% use case for async Middleware

Code Cracked for Code-Splitting + SSR in Reactlandia: React Universal Component + Webpack Flush Chunks

Pre Release: Redux-First Router — A Step Beyond Redux-Little-Router

Redux-First Router lookin Sexy on Code Sandbox

Webpack’s import() will soon fetch JS + CSS— Here’s how you do it today

React Universal Component 2.0 & babel-plugin-universal-import

Tweets and other love are much appreciated. Find me on twitter @faceyspacey

No time to help contribute? Want to give back in other ways? Become a Backer or Sponsor to webpack by donating to our open collective. Open Collective not only helps support the Core Team, but also supports contributors who have spent significant time improving our organization on their free time! ❤