Let’s Build A Web App! Week 28 of my 30 week web dev journey

In which I iron out some issues with my swarm configuration, finalize Ansible playbooks, validate my stage DB, and fix my browser bundle for stage.

Previous posts in this series:

• Let’s Build a Web App! A blog series by someone starting their web dev journey
• Let’s Build a Web App! Week 2 out of 30
• Let’s Build a Web App! Week 3 out of 30
• Let’s Build a Web App! Lo-fi Edition / Week 4 out of 30
• Let’s Build a Web App! Week 5 out of 30
• Let’s Build a Web App! Week 6 out of 30
• Let’s Build a Web App! Week 7 out of 30
• Let’s Build a Web App! Week 8 out of 30
• Let’s Build a Web App! Week 9 out of 30
• Let’s Build a Web App! Week 10 out of 30
• Let’s Build a Web App! Week 11 out of 30
• Let’s Build a Web App! Week 12 out of 30
• Let’s Build a Web App! Week 13 out of 30
• Let’s Build a Web App! Week 14 out of 30
• Let’s Build a Web App! Week 15 out of 30
• Let’s Build a Web App! Week 16 out of 30

(Might have lost proper count here)

• Let’s Build a Web App! Week 18 out of 30
• Let’s Build A Web App! Week 19 out of 30
• Let’s Build A Web App! Week 20 out of 30
• Let’s Build A Web App! Week 21–23 out 30
• Let’s Build A Web App! Week 24 out of 30
• Let’s Build A Web App! Week 25 out of 30
• Let’s Build A Web App! Week 26 out of 30
• Let’s Build A Web App! Week 27 out of 30

Now let’s get back to update on what I’ve been up to in the 28th week of this journey.

What did I set out to do?

I’ve been working on deploying my app for a long long time now — my goal for this week was (again) to finish with that and restart my work on the reader module of moshimoji.xyz. I postulated the following tasks to wrap up deployment:

• validate that the database I’d added to my stage swarm worked with my Django app (potentially initializing some data to get things to work)
• configure my app to connect to the RDS instance in prod and validate that that database worked with my Django app (potentially initializing more data here as well)

On the reader side, I wanted to do the minimum to get to an “MVP” for that module — getting my React components to query first, my GraphQL backend for the assets to pull for a chapter of a series, and then, my Amazon CDN for those assets.

What did I get done?

I’m going to repeat what I did last week and summarize what I tackled each day. But spoiler alert, I STILL didn’t finish with deployment. I did iron out some issues with my compose file for my swarm, finalize a lot of push and deploy playbooks, validate my stage DB, fix my browser bundle for stage, publish some commits, and update my master repo for moshimoji.

Monday: I wrote a playbook or two and deployed my new Compose file for my stage swarm using Ansible. The Compose template had a few small errors, so I fixed those.(+travails/victory log in dev stuff:-Deploy-fails-with-”services, +travails/victory log in dev stuff: Deploy-fails-with-”secrets.DB). Once the deploy of the swarm worked, I checked how the database was playing with the app. Apparently there was a 504 timeout error connecting to the database. I meant to tackle that next, but unfortunately lost my focus and developed something like a light fever, so that didn’t happen on this day.

Tuesday: I picked up where I left off with validating my stage db. The 504 timeout error indicated that there was something wrong with the connection process to the database. I checked the logs and saw a ‘not defined’ error (code mistake). While fixing that, I realized that I was only exposing my docker secrets to the database task, but my Django app attempts to access them as well to get the credentials to make the connection. Oops. Fixed that as well.
After that, I got an error reporting failed password authentication for the database. My first assumption was that I’d forgotten to furnish the needed settings for the initialization of the database, but that wasn’t true — I was. The error message clued me in to trailing new line in the passed value for the user, so I added an task in my push playbook to strip trailing new lines from my secrets after pushing (my text editor automatically adds the new lines, apparently). The password authentication worked after that.
Following that, the next error was about a relation that did not exist. This let me realize that I needed to run a migrate after first initializing my databases for moshimoji, to ensure that the tables for the Django app got created. I composed the playbook for the migrate, ran it, and saw the view render.

Wednesday: Though the previous day I got a view to finally render without reporting an error related to the database, I saw that my login logic wasn’t kicking in as intended. I checked the console and realized that my browser bundle wasn’t configured correctly to set the endpoints for graphql queries and so on. I amended my Webpack config to resolve this.

Thursday: I was getting a type error in the console for a data variable being undefined. This reaffirmed my suspicions that I would need to initialize some data in my stage database after spinup. I created a playbook to get a SQL dump from my development db and apply it to my stage database. This resolved the type error. After this, I made some commits to backup my work (it’d been a while so I was getting increasingly fearful of losing all my work. lol).

Friday: I’d gone for a while without making actual commits for my work now, so I went into each repo and revised my git history into (more) proper commits, pushed branches, and merged pull requests. I created a new submodule for nginx since I was separating that out from my backend repo, and updated each of the subrepos in the master moshimoji repo.

What didn’t I get done?

• I didn’t get to setting up my prod swarm, which includes the following steps:

—

1. updating a record to elastic load balancers for prod
2. checking nginx logic
3. adding prod endpoint-setting logic in reactql config => new frontend build
4. updating prod config for backend=> new backend build
5. adding prod manager to ansible hosts
6. pushing files like compose and secrets to prod swarm

—

• I also noticed some error messages around failed retrieval of static assets for misago, which would explain the absent styles for my Misago view, but didn’t get around to figuring out the issue.
• Also my login modal has a funky render which I didn’t get to debugging.
• Finally, I didn’t get to developing on my reader module further at all.

What went well?

• The time that I did work, I felt like I made steady, if gradual, progress and was able to intuit fairly quickly what was the issue behind each error.
• Giving things another shot after taking a break — I was able to sort out the Ansible behavior that stumped me so hard last week without too much issue this week.
• When I found some odd hours to work, I took the chance to work, which is great, because I’m normally not that comfortable working in fragmented blocks of time.
• Using git reflog to recover when I deleted a branch before realizing it was the only branch with some commits I needed. Thank you git reflog.

What was difficult?

• It’s a goal of mine to work more consistently, but this week there was a lot of squandered time being inefficient in the mornings.
• A lot of stuff I worked on this week were errors due to details I just forgot about or missed, like the need for a migration, the need to set the STAGE process.env var for my production package, and etc. I don’t think I take that much longer to handle it after seeing the error as opposed to before, but something about taking care of these details only after trying deployment feels messy and wrong. Probably because it means I have this prolonged period of ‘fixing’ things after deployment.
• I think ultimately having my Ansible playbooks set up saves me time, but I did feel the sunk time each time I took wrote up the playbooks instead of just issuing the build, push and deploy commands manually. If I wanted to prioritize validation of my swarm setups as much as possible, skipping the playbook composition for now would have allowed me to move a little faster.

What would I do differently next time?

The biggest thing from this week that I would have changed was just how much I worked.

A few cool things I learned about

• a little more about Docker builds — how they create a build context (directory) from the specified directory, how all paths in the Dockerfile only work relative to that build context, and the need for the Dockerfile to be present in whatever context you chose for the build
• Docker secrets are immutable, which I suppose makes sense, but also makes it tricky to update your secrets
• DJANGO_SETTINGS_MODULE
• options to modify ansible error handling in tasks
• git reset — soft, the process for updating submodules; git fetch to fetch remotes; git checkout [branch name] — remote

Misc links for your browsing pleasure:

• My twitter: https://twitter.com/ronachng
• My Github repo for kusamochi: https://github.com/ronachong/kusamochi/commits/master
• My Github repo for moshi-moji:
  https://github.com/ronachong/moshi-moji
• My Paper doc for documenting some travails and clearups during development:
  https://paper.dropbox.com/doc/travailsvictory-log-in-dev-stuff-urw9F3oeUadSzjULXWYiM
• My Paper doc for brainstorming/planning my web projects:
  https://paper.dropbox.com/doc/different-ideas-mnesHVM2GY2Fwb05iN2fW
• My Paper doc for logging my Google queries (obsessive, I know):
  https://paper.dropbox.com/doc/resource-log-rOGTRd8hygvwJ4NCGglpm