GSoC’21: Scan8 Application Architecture

This blog post marks the onset of the coding phase. We have now finalized the application architecture, which I will discuss in further detail here.

The complete application has two major parts:
1. Full-stack application development
2. Application deployment

I will be working on the full-stack application development side of things during GSoC’21. The idea is to develop a single-coordinator and single-node model and then scale it in the deployment phase using Kubernetes.

Application architecture

The application consists of various blocks that work together to give a full-fledged antivirus scanning service.

The Front-End is exposed to the end-user. It facilitates the process of submitting new scans and helps track the progress of currently running scans. Flask is a python based micro web back-end framework that listens to requests on different routes and appropriately responds. MongoDB database is used to store relevant information corresponding to each scan.

The coordinator defines specific tasks, including interaction with ClamAV, and adds jobs to the Redis Queue. Finally, the node runs a Redis Queue worker to perform enqueued jobs and store the scan results in storage.

Application workflow

The user submits a scan using the HTML + CSS user interface, and the Flask micro back-end handles requests concerning the same. Flask then hands over data to the coordinator. Next, it updates the database and defines the job added to the Redis Queue (RQ). A node runs an RQ worker to dequeue jobs and execute them. It also updates the MongoDB database regularly, from where the job completion statistics are supplied to the UI. On scan completion, the node stores the results to storage and updates the scan status in the database. If a job fails, it is resubmitted to the queue.

Stay tuned for further updates :)

Project: Scan8

Feel free to connect with me on LinkedIn and check out my GitHub profile.