Migrating an app to Next.js: Lessons learned

Originally published at jamischarles.com.

I’ve spent the last few months building a side project: https://frontendrocket.com: a job board for frontend engineers.

Initially I just wanted to launch something really fast, so I chose a very boring setup:
Server:
Node.js + Express.js + EJS for templating
Client-side:
VanillaJS

Why migrate to Next.js?

EJS was becoming really unwieldy, so I looked around and asked people what they liked to use for rapid prototyping (any language, any stack). A number of people said Next.js.

I knew it was probably overkill but I wanted to try it out on a real project to fully explore how it handles the complexity. This post is about Next v6.0.3.

The Verdict

The verdict: Next.js was overkill for what I needed. I did stick with it for several weeks and learned a lot of things about building a universal react app. Let’s dive in.

What is Next.js exactly? What problem does it solve?

Next.js home page

It took me a few tutorials to wrap my head around this. Is Next client-side only? Does it include an express server? It’s labeled as a framework, but I think it’s clearer to call it a frontend framework.

Next is a client-side framework that happens to have a server-side component to allow features like HMR and SSR. You can definitely combine it with your [server] app.
Source

The good news is that you could use it with Rails, Express, or other backend frameworks.

I’ll be primarily discussing the SSR (Server-side rendering) use case in this post.

What does “Universal” mean?

Previously termed “Isomorphic”, Michael Jackson suggested we change the name to Universal and it caught on. This basically means that you share much of the same template and other code between the server (node.js usually) and the client.

Why would I want a Universal app?

A Universal app is one option when you have a client-side SPA (single page app) but don’t have server rendered templates yet. SEO generally favors server rendered content. More on that here.

Should I make my app Universal?

For the majority of people, I think the answer is no. The complexity cost just isn’t worth it.

First steps with Next.js

Next.js Simple tutorial

There is a very simple Next.js tutorial: Getting Started — Learn Next.js

It’s great and it shows how the simple use cases work really nicely. Next.js was inspired by the simplicity of using a php file

The ease-of-use of PHP is a great inspiration. Source

They really nailed that part. It’s kind of amazing that you drop JSX template code into the /pages folder and the routing and everything just works. Magic ✨.

Any JSX templates in /pages automatically just work with client-side routing. Wow.

So I dove in and started converting my express app.

Stumbling blocks

When you move beyond the simple use case is where things get a bit hairy.

How do I get the server-side routes?

When you run Next locally, the initial page works, and client side routing works great. But you can’t refresh any of the pages beyond the index page.

If you want server side routes, you have to add a server. In my case I used the simple express server.js file the tutorial showed.

A simple express server for use with Next.js

Previously we started the app with the next command. Now we use node server.js instead.

We can now refresh the pages and they still work. Great! Onward.

First problem: How do you pass props to the the views?

There are several ways to pass the server properties to the views:
1) You can use req.query and pass that to the client. This will only be available to the template on page refresh. Client side routing requires you pass these props along with the <Link> tag in this way <Link href={{pathname:'/second', query:{title:'Title two’}}}

This is the first way we can pass props from the server to the client

This approach has a few downsides. I don’t want to pass all the data to every <Link> tag, and everything in props.url.query will get rendered to the html document. Lots of server props = bigger html file.

2) You can fetch it from getInitialProps(). This means you need to use a proper React component instead of a functional component.

This suffers many of the same problems as 1)

3) You can fetch it in getInitialProps, and then make a fetch call on the server and the client

If you want to cache these ajax calls, you could store the result in sessionStorage, and make the call optionally from componentDidMount().

All this is extra work that increases the lines of code and the complexity of the app.

Second problem: Page size

I have a list of 200 job listings on a page. With my server rendered EJS version, this was ~500kb unminified. When I switched to next.js it was ~1MB primarily because all the query data that was passed to the views was being written to the html in addition to prior markup.

All the props are written to the html :(

I found several GitHub issues that discussed this problem. I couldn’t see any good solutions to this in these issues.

My solution: Lazy load data if at all possible.

Third problem: All the data needs to be loadable from client and server.

First you have to ask yourself “will all pages be universal, or only some of the pages?” You can pick and choose. Keeping some of the pages accessible by page refresh only will simplify things.

For the universal pages, you need to ensure that all the data can be loaded from the server or the client.

This is where the biggest mental shift comes in:
I discovered that I needed to move most or all of my data that need to be passed to views to separate routes.

This shifted the prior mental model I was using. Previously the express route handler would dictate “when this route is called, fetch this data, render this view, and inject these props into to the view template.

Now it changes to “when this route is called, render this view. The view then says “I need this data, fetch it from these data routes”. On first page load this data fetching happens server side. Navigating to the page after that makes it an ajax call (if it’s a universal page).

We remove the data from the template routes

Now, an ajax call to i.e. /data/listing/remote will return all the data for remote listings as a JSON object.

I think I like this new mental model better, and sets the stage nicely for Relay or GraqhQL.

Lazy loading data

A large portion of my page size bloat was coming from the “description” field which is hidden behind expandable rows on first load anyway. This is a perfect candidate for lazy loading.

Collapsed job listing
Expanded job listing (more details not shown)

I spent an hour or so and changed my data structures so I could split out the description data (I think GraqhQL would have been nice here).

So now, on page refresh, or on client side redirect, we fetch all the job descriptions for current location after the UI has rendered, and store it in sessionStorage:

Fetch the job descriptions, and store in sessionStorage

Faster? Definitely. Initial html page size shrunk from ~500kb to ~135kb. After that, the JS bundle Next sends is ~155kb (unminified dev build). After the UI has rendered the /details ajax call returns with 208kb in data. We’re loading the same amount of data, but (thanks to lazy loading) the page loads noticeably faster. It’s also more complicated and added a whole lot of logic. I didn’t love having to do this, but was committed to trying it out at the very least.

Deploying

Figuring out how to deploy it was a little tricky. I was previously using pm2 to deploy my app to DigitalOcean and after deploy was running npm install && npm start. After some trial and error I figured it out: npm install && next build && npm start. Arguably you could build it locally and check that into you git repository.

Gotchas

There were some things that tripped me up.

Script tags

Script tags like Google Analytics generally shouldn’t be rendered in React Components. With universal Next components you have the potential of rendering it once on the server and many times on the client.

Solution: Drop a JSX template file in /pages/_document.js. This is the base template for all server rendered pages. You can safely add script tags here. If there are script tags you want to load conditionally, or only want to load on certain pages, you have to add that logic here as well.

Page refreshes vs client side nav

You can mix page refreshes and client-side navigation. If you want to have a page refresh on navigation, use a plain <a href=""> tag, instead of the <Link> component Next provides.

<Link> vs <a>

Bonus: <Link>lets you maintain scrolling between pages and makes navigation feel much faster.

<Link> tag confusion

<Link as={}> vs <Link href={} was another big point of confusion for me:

TheAs attribute is the display name in the browser. The href attribute refers to the physical template file in /pages that will be rendered. Anything in href={query} will be passed to `query` in getInitialProps when the component is client-side rendered. Confused yet? State on the client is hard. Universal apps make this even more complex.

Server rendered components vs client rendered components

Server rendered components vs client rendered components can be confusing. This took me several iterations to figure out. getInitialProps() is called server side only on a page refresh. When a <Link> is followed (client-side nav), getInitialProps() is called client side. That means you need to ensure that it can safely be called both ways. componentDidMount() is only called client side.

Debugging errors

Errors were a bit tricky to debug. Stack traces would occasionally show errors in the wrong place. Not sure if this React, Next, or a plain JS issue.

Surprises

Rewriting all the template logic from EJS to JSX took a surprising amount of time. I think that’s primarily because EJS and JSX are very different mental models.

I realized how much I hate having looping logic mixed in my template. I really love how JSX makes it easy to loop before the return statement:

HMR. To me, not worth it. It doesn’t always work. Sometimes you still need to refresh. As a result I don’t really trust it and end up manually refreshing most of the time anyway.

The biggest shift I had to undergo was going changing my mindset about what a universal app actually is and what it isn’t. What I was actually looking for was to replace EJS on the server with JSX. You can do that. But that’s not what Next.js is.

Big takeaways

  • Writing Universal apps is hard. With a client-side SPA you take all the server state and move it to the client. That increases complexity. With a Universal app, you’re keeping all the server state, and you have to keep it in sync with the client state. This is more complicated than both doing a server rendered app or an SPA (single page app).
  • For many apps, Universal is overkill. We all love the promise of reusing code from the server and the client, but in my experience the added complexity is rarely worth it.
  • Next.js is arguably the simplest way to write a universal app. It’s still pretty complex.
  • I found migrating a server side app to Next.js is too much work. Maybe it would be easier to first migrate to a react.js app, and then try to make it work on the server.
  • I’ll give Next.js another shot in the future, but I’ll start with Next.js on day 1, rather than migrating to it.

What was your experience with Next.js?