Notekeeper: A Journey into CI/CD
Notekeeper is an open source web application built using Python and Flask. It features account creation, writing, storing, editing, tagging, and searching notes. The web application can either be deployed locally or on a cloud server.
This project is a part of Byte Size’s Final Project in the Strategio Enterprise Simulator: to create and run a CI/CD pipeline for a web application. It is built upon A-Simple-Note-Taking-Web-App by Omkar Pathak.
Users
When considering the goal of our project was, we considered who our end-users would be. To that end, we created some personas that allowed us to center our focus.
My name is Kris and I am a home cook. Recently I’ve realized that I have too many recipe books and am having trouble remembering which book contains the recipe I’m looking for, so I want a searchable database of all my favorite recipes. By having one central location where I can find all my favorite recipes regardless of their origin (for example, one place for chicken recipes instead of flipping through one book at a time for chicken recipes), I will be able to focus more on what I enjoy, which is cooking.
My name is Sam and I am beginning my first year at university. Keeping track of all my notes and assignments for all my classes is challenging. I’ve used physical notebooks in high school but it’s a pain to carry everything around and I want to be able to search through my notes quickly to find notes on specific concepts that I need to study more. I want to be able to organize my notes in one convenient location so that I can study efficiently and effectively and get good grades in all my classes.
Features
Adding a new note
Markdown supported entry field, live preview, entry for note title, and tag(s) selection field.
Viewing a note
See note detail and contents, with options to edit or delete the note.
Viewing all notes
See a list of all notes stored.
Searching notes
Search for keywords in note titles to filter results.
Adding a tag
Viewing all tags
See a list of all the created tags.
Viewing tagged notes
See a list of all notes tagged with a specific tag.
Profile Settings
The profile settings page displays details about the user, an editable user email field, and links to the pages that show all notes and tags stored on the account. Another search bar for the notes appears here for convenience.
Video Demo
If you would like to see Notekeeper in action (as well as a teaser of the toolchain and architecture of our project which is discussed later), please check out the video below.
Technologies Used
- Python 3.8.14
- Flask 0.12.2
- SQLite
- Pytest
- Terraform Cloud
- AWS EC2
Architecture
CI/CD Pipeline
The CI/CD pipeline we used for this project is shown here, with each tool of our toolchain separated by which phase of the DevOps cycle it’s utilized in.
- Initial planning, requirements, and task assignments were conducted through Notion
- VS Code was our primary IDE for coding and development
- The GitHub repository we were provided acted as our version control system
- In addition to GitHub, we leveraged GitHub Actions for our Build, Test, and Release model
- A workflow is triggered on a push to the main repo, which builds the Python code in a newly provisioned Ubuntu virtual machine
- Pytest is then invoked to run tests against the program
- If the build and test succeed, GitHub Actions begins another job to provision an AWS EC2 instance
- This is done through Terraform Cloud
AWS Deployment
Here is a closer look at the AWS Deployment.
- The triggered workflow automates the Terraform setup — installing the CLI — and executes the typical build deploy commands (terraform init, plan, apply)
- Inside our repo, we have a main.tf file which holds the Terraform configuration to apply
- Using this, Terraform Cloud provisions the AWS Cloud Provider to generate an EC2 instance contained in an associated VPC, deployed in the us-east-1 region
- Once the EC2 instance is live, we have a public IP and URL that can be accessed through the internet
- Terraform Cloud also provides a destroy plan that can be queued to tear down the automatically built AWS resources
Challenges
The initial codebase we used as a basis for Notekeeper was last updated nearly 3 years ago. It was built for an older version of Python and used then up-to-date libraries. Which meant that certain code had been deprecated and some dependencies broken. However, with some troubleshooting, we refactored the codebase to get a working version up and running.
Both GitHub Actions and Terraform Cloud had learning curves to utilize them effectively. Thankfully, there was an abundance of official documentation and resources we could consult with to correct our mistakes. We had to fail fast and learn in order to better our understanding.
Future Development
As Notekeeper was developed over a period of approximately 2 weeks, as a team we had to keep that scope in mind. It’s thanks to that consideration that we were able to deliver a product at the end of the period. Still, we did think about what we would like to do if we were given additional time.
Refined Tag Search
Currently, notes can only be filtered by one tag at a time, despite the fact that notes can have multiple tags. We would like to implement a multi-tag search functionality that takes advantage of this fact.
Sharing Notes
Users can only see notes that they’ve made themselves as the notes are associated with their accounts. While this is the desired behavior, we believe adding a feature to share notes between users would be positive for the user experience.
HTTPS & Custom URL
Since we are hosting Notekeeper on an EC2 instance, the public URL is an AWS derivative. If we could secure a website address such as “notekeeper.com”, we could more easily advertise it and direct users to the application. Additionally, the application uses HTTP internet connections but we would like to move towards more secure HTTPS connections.
Database Separation
User account data is currently stored in an SQLite database that lives on the same EC2 instance as the Noteekeper application. In accordance with best practices, we would like to transition the database to a separate instance, possibly using an AWS database service such as RDS.
