Creating a sitemap generator for Next.js
From sitemap creation to search engine index request
In this story, I’d like to share the search engine optimization — that was done while converting the Volla bridge page which was written in Vue.js into Next.js — and how I created a dynamic sitemap generator for Next.js.
1. Create sitemap generator script following Next.js folder structure
Next.js automatically creates URL following folder and file name in the pages directory. So, I’m going to write the XML generating script, that exclude the special file names of Next.js (_document.js, _app.js etc.) among all the folder and file names in the pages directory using globby.
First, create a scripts folder in the root directory, and write the following code named sitemap-common.js.
Executing the above script creates xml file as below(example).
2. Writing sitemap generator script for external API
As in the example code above, it’s relatively easy to create sitemaps for static pages. However, for dynamic pages(e.g. pages with userId), we need to write a script that creates a sitemap in a slightly different way (in the example code, I used JSONPlaceholder’s API).
I wrote the following code — named sitemap-posts.js.
Executing the above script creates xml file as below(example).
3. Writing script that compresses all sitemap files created above into gzip format
Sitemaps that compressed in gzip(.gz) format can be used in the same way as xml format sitemaps, but in a smaller size. Using zlib, I’m going to compress all the xml files created above into a gzip format.
The above script compresses all the xml files created in steps 1 and 2 into .gz format.
4. Writing sitemap index generating script for the sitemap files which created by the above methods
Providing multiple sitemaps to search engines (Google Search Console, Naver Search Advisor, etc.) requires sitemap index file.
I created each sitemap separately for various dynamic URLs such as /seller/[_id], /product/[_id], and /video/[_id]. To submitting a sitemap to Google Search Console, etc., a single sitemap must be submitted, so I wrote the following script to create a sitemap index file.
Executing the above script creates xml file as below(example).
5. Writing a bash script that creates a new sitemap for each master deployments, and a script that pings the sitemap to Google Search Console from GitHub Actions
# yarn sitemap
$ cd public
$ rm -rf sitemap
$ mkdir sitemap
$ cd ..
$ cd scripts
$ node ./sitemap-common.js
$ node ./sitemap-posts.js
$ node ./sitemap.jsIf you want to make Google Search Console reindex your pages, just add one line below at the end of the script.
$ curl http://google.com/ping?sitemap=http://website.com/sitemap.xmlI wrote a new bash script like below to create sitemaps in xml format, compress those sitemaps into gzip format, and remove unused xml files automatically.
Afterwards, I modified the workflow file to run the above scripts in GitHub Actions on master deployment.
# Create a sitemap and submit it to Google.
- run: yarn sitemap
name: ping sitemap
