Background File Download With Go & React

Today we will be building a simple web server in Go that streams the content of PDF files to the client which will be a react application. The react application will collect these chunks and show progress on what percentage of file has been downloaded. First, let’s start with the go application.

Go Server

The backend application will have folder structure as follows

1. files folder — contains few pdf files that we will be streaming to client
2. main.go — go functions that serve the client

Inside files we have some pdf files of varying size from 1MB to 99MB since GitHub has size limitation of 100MB for a file inside a repo.

We will have 2 routes handled by our server inside main.go. The first route will send a slice of FileInfo type which contains file’s name and its download link, both of which will be string. The second route will stream the content of files based on download link mentioned earlier.

type FileInfo struct {
 FileName     string `json:"file_name"`
 DownloadLink string `json:"download_link"`
}

Now let's move towards implementing the functions that will handle the route. For the route that sends info of files available for download, we will implement a function inside which we will call another function that will read the files present inside Files directory and return slice of FileInfo. The download route will be the same for every file with the difference being in query. So the later function will look as below.

func GetFileInfo() []FileInfo {
 var files []FileInfo

 fileList, err := ioutil.ReadDir("./files")
 if err != nil {
  log.Fatal(err)
 }

 for _, file := range fileList {
  fileName := file.Name()
  files = append(files, FileInfo{
   FileName:     strings.Split(fileName, ".")[0],
   DownloadLink: fmt.Sprintf("/pdf?file=%s", fileName),
  })
 }

 return files
}

The function that handles the initial route will simply return the received slice as JSON. However, we will face a CORS issue since we are developing front and back applications locally. So we will setup a function that accepts requests from any origin. The function will write the header for the same as follows.

func enableCors(w *http.ResponseWriter) {
 (*w).Header().Set("Access-Control-Allow-Origin", "*")
}

Combining these functions inside the route handler function looks as below.

func Info(w http.ResponseWriter, r *http.Request) {
 enableCors(&w)
 files := GetFileInfo()
 w.Header().Set("Content-type", "application/json")

 marshaled, err := json.Marshal(files)

 if err != nil {
  log.Fatal(err)
 }

 w.Write(marshaled)
}

Now moving towards the next route that handles the streaming, we will enable the CORS as first step. Then on the basis of query received, we will read the file. The query file simply includes the file name so the file’s path will be /files/FILE_NAME_FROM_QUERY. Next, we will grab some file info which will be its total size to be passed in the header so that the frontend application can tally a chunk of data received and total data size to be received which will be useful to show download progress. Now we read the file using ioutil the package, convert this to buffer and simply write to response making the function as below.

func PDF(w http.ResponseWriter, r *http.Request) {
 enableCors(&w)

 fileQuery := r.URL.Query().Get("file")

 FILEPATH := fmt.Sprintf("./files/%s", fileQuery)

 fileStat, err := os.Stat(FILEPATH)
 if err != nil {
  fmt.Println(err)
  os.Exit(1)
 }
 // get the size
 size := fileStat.Size()

 // grab the pdf file and stream it to browser
 streamPDFbytes, err := ioutil.ReadFile(FILEPATH)

 if err != nil {
  fmt.Println(err)
  os.Exit(1)
 }

 b := bytes.NewBuffer(streamPDFbytes)

 // stream straight to client(browser)
 w.Header().Set("Content-type", "application/pdf")
 w.Header().Set("Content-Length", strconv.Itoa(int(size)))

 if _, err := b.WriteTo(w); err != nil {
  fmt.Fprintf(w, "%s", err)
 }

 w.Write([]byte("PDF Generated"))
}

So by now we have implemented functions to handle two routes and only thing is remaining is setting up the server. Inside main function we will initialize a new server and define the function handlers implemented earlier for intended routes and listen to the server in a port.

func main() {
 mux := http.NewServeMux()
 mux.HandleFunc("/", Info)
 mux.HandleFunc("/pdf", PDF)

 http.ListenAndServe(":8080", mux)
}

And here we go, run our server using go run main.go and our server is up and running ready to serve the frontend requests.

Client React Application

Now let's move towards creating a frontend application with react that will be making requests to the server.

npx create-react-app background-download-front

Briefly talking about the flow of frontend application, it will be as follows

• first, call the API route that returns the list of file information
• Based on this response it will show the file names and button to download the file
• On download, get a stream of data store received bytes in a variable and compare with the total to be received
• Based on the above variables show a download progress bar
• The progress bar will be based on the download state which is either not downloading, downloading, or download completed which will be defined as constant

So let's first create a constant.js file which will include API link and download states mentioned above

// constant.js
export const API = "http://localhost:8080"

export const NOT_DOWNLOADING = {
    value: 0,
    label: "Download"
}
export const DOWNLOADING = {
    value: 1,
    label: "Downloading"
}
export const DOWNLOAD_COMPLETE = {
    value: 2,
    label: "Download Complete"
}

Now lets create a component DownloadPage which will call the api to get list of files and based on response call another component that handles its rendering and download functionality.

// DownloadPage.js
import { useEffect, useState } from "react"
import { API } from "./constants"

const DownloadPage = () => {
    const [downloadList, setDownloadList] = useState([])

    useEffect(() => {
        const fetchList = async () => await fetch(API).then(res => res.json()).then(data => setDownloadList(data))
        fetchList()
    }, [])


    return <div className="download-list" >
        {
            downloadList?.map((d) =>
                console.log("download component will be called from here for ", d)
            )
        }
    </div>
}

export default DownloadPage

As we can see above, we have just logged our files info as we are yet to implement the component that performs downloads. We will name the component DownloadFile which will receive fileName and fileLink as props. So let’s create the most awaited component and start with rendering aspect.

// DownloadFile.js
import { useState } from "react"
import {NOT_DOWNLOADING} from "./constants"

const DownloadFile = ({ fileName, fileLink }) => {
   const [downloadState] = useState(NOT_DOWNLOADING)

    return <div className="single-file-container">
        {fileName}
        <button className="download-button">{downloadState.label}</button>
    </div>
}

export default DownloadFile

Nothing much fancy here, just showing the file name and a button. We will keep track of download state to show if file can be downloaded, it is being downloaded or download is complete. The initial state will be not downloading so user will see Download as button label. There will be some styling aspects added to the components for which we won’t go into much detail but the css file will be shown later.

Further, we need to handle the download function which will be triggered after clicking the button. Here are the things that we will be doing when download is triggered

• Set the downloadState to DOWNLOADING
• Start fetching data from the download route
• Grab the file’s total size present in Content-Length header
• Read the response body
• Store current time inside a variable to later determine time taken to download the file
• Inside the continuous loop receive the value & done properties from reader
• if done is not true then
  - we will push the received value in an array that holds all bytes received
  - add received size to variable that holds total data received so far
  - Calculate the progress in % by dividing received size by total size to receive and multiplying by 100
  - store the calculated progress in a state
• if done is true then
  - Store current time inside a variable
  - Determine the time difference between start & end time and store it inside a reference
  - create a blob using an array of chunks received
  - create a download link using this blob
  - create an element with a tag and download link
  - trigger click for downloading the file
  - set downloadState to DOWNLOAD_COMPLETE
- set a timeout function for 4000 ms after which
  - set downloadState to NOT_DOWNLOADING
- set download time reference to 0

Above mentioned algorithm is what will download the requested file. Now what’s remaining is to show the download progress bar. So inside the rendering component, if the download state has some value i.e is downloading or download is completed the we will show an element. Inside this element we will have another element whose width is set to progress state’s value and so it will be filled up to progressed %. We can also add another element to show progress state as its actual value as label. In addition, we will show the additional download button’s label based on download state i.e. show time taken to download if download state is completed. So the DownloadFile component will look as below.

import { useRef, useState } from "react"
import { API, NOT_DOWNLOADING, DOWNLOADING, DOWNLOAD_COMPLETE } from "./constants"

const DownloadFile = ({ fileName, fileLink }) => {

    const [downloadState, setDownloadState] = useState(NOT_DOWNLOADING)
    const [progress, setProgress] = useState(0)
    const downloadCompleted = useRef(0)

    const handleDownload = async (fileName, DownloadLink) => {
        setDownloadState(DOWNLOADING)
        const response = await fetch(`${API}${DownloadLink}`, {
            "method": "GET",
            "headers": {
                "Accept": "application/pdf"
            }
        })
        if (!response.body) return

        const contentLength = response.headers.get("Content-Length")

        const total = typeof contentLength == "string" && +contentLength

        let receivedLength = 0

        const reader = response.body.getReader()
        const chunks = []
        const start = Date.now();
        while (true) {
            if (total > 0) {
                const percent = 100 * receivedLength / total
                setProgress(+percent.toFixed())
            }
            const { value, done } = await reader.read()

            if (done) {
                const end = Date.now();
                downloadCompleted.current = (end - start) / 1000
                const blob = new Blob(chunks)
                const url = URL.createObjectURL(blob)
                const a = document.createElement("a")
                a.href = url
                a.download = `${fileName}.pdf`
                a.click()
                setDownloadState(DOWNLOAD_COMPLETE)
                setTimeout(() => {
                    downloadCompleted.current = 0
                    setDownloadState(NOT_DOWNLOADING)
                }, [4000])
                break
            }
            chunks.push(value)
            receivedLength += value?.length
            console.log("value ", value?.length)
        }
    }

    return <div className="single-file-container">
        {fileName}
        <button disabled={downloadState?.value} onClick={() => handleDownload(fileName, fileLink)} className="download-button">{downloadState.label} {downloadState?.value === 2 && `in ${downloadCompleted.current}s`}</button>
        {downloadState?.value ? <div className="progress-bar">
            <div className="completed" style={{ width: `${progress}%` }} />
            <span className="progress-indicator">{progress}%</span>
        </div> : <></>}
    </div>
}

export default DownloadFile

Now that the DownloadFile component is implemented, we can call it inside DownloadPage component and it will look as below.

import { useEffect, useState } from "react"
import { API } from "./constants"
import DownloadFile from "./DownloadFile"

const DownloadPage = () => {
    const [downloadList, setDownloadList] = useState([])

    useEffect(() => {
        const fetchList = async () => await fetch(API).then(res => res.json()).then(data => setDownloadList(data))
        fetchList()
    }, [])


    return <div className="download-list" >
        {
            downloadList?.map((d, i) =>
                <DownloadFile fileName={d?.file_name} fileLink={d?.download_link} key={i} />
            )
        }
    </div>
}

export default DownloadPage

Start the server using the command.

npm start

The final result is shown below. We can simulate the application in a Slow 3G from the network tab to see the progress bar moving slowly in case it's not noticeable on normal internet speed.