Rebuilding Underlost.net
An introduction to the design, development and deployment process of the latest version of my site.
With parts 2 and 3 being a little more technical, I was originally going to split this up into multiple posts, but decided to leave it as is. I tried to keep explanations and details in the development/deployment sections as easy to follow along with as I could, so even those unfamiliar with the tools and services I used could still get the gist of what I was trying to accomplish. Originally written May 12th, 2014, on underlost.net.
Introduction
Designing underlost has always been difficult. It’s always been more than just another website to me. It’s a part of me. It’s my online persona. For better or worse, I never really hid my identity online. I always went by ‘underlost’. For as long as I can remember, my real name and pseudonym were synonymous. I became okay with this pretty fast. So when it comes to my site, it’s almost a reflection of myself. It’s like showing the world who I think I am. And quite frankly, putting yourself out there like that can be pretty scary.
It can also be pretty rewarding too. Whenever I rebuilt underlost.net, I would scrap all the old content— writings, images, designs, everything. I was never really too worried about preserving it. It would continue to live on in backup directories and forgotten databases. Whenever I would rebuild, it was like a phoenix rising from the ashes. It was therapeutic. It would force me to self reflect. Who was I now? What have I accomplished so far? What do I need to work on? In high school it acted as a sort of journal. I blogged before ‘blogging’ was a real thing. I would talk about random school sports. I would share the source code that would generate the passcode to let you bypass the school’s annoyingly stupid security program that ran on almost every computer, ‘Fortres’.
In college, the site acted as sort of a public drop box, posting pictures of our World of Warcraft guild clearing out Molten Core or the gates of Ahn’Qiraj, or using it to host mod files I had written for Unreal 2k4 and Knights of the Old Republic II. But after college and the closure of my startup ulost, I was kind of lost. I really didn't know what I wanted to do next. That sense of being lost was reflected on the site. For a while, I went through literally dozens of design ideas for the new site, but nothing really stuck. I jumped around different content systems. Movable Type to Drupal to WordPress. I even started writing the first incarnation of my own CMS, Replica. Partially because I had convinced myself I needed to write my own system that could give me the freedom to combine my writing and design skills for posts. When in reality, I was just looking for excuses to not write.
I put the site on hold for a while. I won’t really get into it in this post, but after doing some soul searching, and getting incredibly lucky to work with some fantastic people, I felt like I finally had a sense of what I wanted to do, of who I was and what I wanted to be. For the first time in a long time, building underlost.net didnt feel like a chore. I knew what I wanted it to look like and kind of content I wanted. Most importantly, by the time I was done, I actually liked it, and didn't want to start over.
Development
One would think that because I wrote and maintains my own CMS, I would have just used that to power the site. For a while, I did use it. However, having a full blown content management system was a little out of place for a site my size. It was simply overkill. Most of the site doesn't get updated often. Really all I needed was a static site to host a few dozen HTML files. I didn't need anything too fancy; nothing dynamic, and surely not a database to house all the content.
Enter Jekyll. For those unfamiliar, Jekyll is a static site generator written in Ruby. Coming from primarily a python background, I could have easily just used Pelican instead, but I feel it’s important to not limit yourself to using just one language to solve everything. Plus, I've had positive experiences with using Jekyll, deploying sites on Github in the past. It also uses a liquid template system, similar to what Django (and Pelican) uses for it’s own templates, so naturally I felt right at home.
I like to think my jekyll setup is pretty straight forward. I’ve managed to avoid any hacky solutions, while taking advantage of some of the new features introduced in version 2.0.0. The most notable feature being collections. The original plan was to just use YML to keep track of the selected works area, but with collections, I was now able to mimic the way _posts work, creating a set of markdown files with the YAML front end data stored in each post, rather than a single ugly file. Writing about it may not sound like a big deal, but for anyone who’s used Jekyll prior to v2 will know how frustrating it could be to copy the way _posts worked for other parts of the site without extra plugins.
Other than that, like I mentioned, underlost.net is a pretty standard jekyll based site. I don't use any plugins other than the official sitemap.xml generator. For organizational purposes, I do keep all the public content in a separate subdirectory, which causes some issues when trying to deploy to github, but I’ll talk more about that shortly.
7/7/2014 Update: Since writing this, while I do still use Jekyll, I've moved to Medium to publish content. Another advantage of Jekyll is the fact I can use the backup tool here on Medium, then mirror the html files easily on underlost.net, in the event medium ever goes down, or vice versa.
Deployment
Not really being into the Node.js hype when it first came out, I dismissed the vast uses of it. It was just ingrained in my mind that Javascript didnt have any use other then for websites. It wasn't until I learned to use Node for work had I realized the capabilities of it. Node.js isn’t just a server backend, but also a tool for front end development. This is where something like the Grunt ecosystem comes in. Grunt, as it mentions on it’s website, is a task runner written in javascript. With it I've managed to automate some of the more tedious jobs, like image optimization and CSS compilation. To be more specific, with a single command, Grunt will execute several jobs for me:
- Compile LESS to CSS
- Image optimization/compression for JPEG/PNG formats
- Run autoprefixer on CSS to add vendor prefixes to rules where needed
- Compile all specified javascript files into a single file
- Minify CSS/Javascript
- Run validation tests on HTML generated by Jekyll
- Run validation tests on JS using qunit.
- Optionally deploy the compiled ‘_site’ directory to Github with git
Finally being happy with the finished results of the redesign for my site, it was time to make it live. But where? I initially relied on Heroku with a modified buildpack and Unicorn/memcache set up. While Heroku is a great service, it can also get quite expensive once you start scaling past their first free ‘dyno’. After looking at benchmarks, it’s also questionable whether or not Memcache even makes any difference for a static site. Since I keep track of the site with git, Github Pages would be the ideal free solution, but they don't allow custom plugins, nor an alternate “source” directory inside _config.yml because all builds run in safemode. I could have made some changes to my site structure to make it work, but I didn't really want to have to to do that. I also wasn't confident in GH Pages ability to deliver pages in a timely manor.
I settled with using Amazon’s S3 and Cloudfront services, with Heroku acting as a staging area. I'm still using my initial setup of Unicorn and Memcache, but I've scaled it down to one dyno to stay on Heroku’s free plan. This way, I can test across multiple browsers and devices more easily than I could locally, while also making sure a deploy is successful before it reaches the live site. But really, the only difference with this new setup is that I'm also using the wonderful s3_website gem on Heroku instead of using it locally. Once deployed to Heroku, I use their CLI to execute an s3_website push to the S3 bucket.
The beauty in this, is rather than storing my credentials in a text file or locally as env variables, they're saved with Heroku, allowing me to work on the site from any machine. Whether it’s more secure that the credentials are stored locally or ‘in the cloud’ (I'm fully aware of the irony of storing cloud credentials in the cloud), but in this case, to me, the ease of access outweighs the threat to the site’s security.
So there you have it. The rebuild of underlost.net. If you haven't noticed by now, I never really said that I'm done with the site. The truth is, I'm a perfectionist, so I will never see my site as ‘complete’ or finished. Like life, it will always be a work in progress and I'll continue to tinker on it and let it grow. It will continue to be an extension of me, even if it is only a virtual one.