Supercharge your Development QA with Heroku Pipeline and Constant Integration
At kohactive we have used a variety of tools and practices to improve our Q&A over the last year.
To start we’ve added a rspec test suite for many of our projects. We’ve also adopted TDD for the api side (rails), for the majority of our new projects. For frontend we have a light weight sanity checking layer in capybara to make sure happy path user flows work.
Building on that we wanted to reduce our dependence on one or two people to deploy our app. After investigating solutions, we immediately fell in love with the heroku pipeline.
Iteration 1, we setup a Production and Staging application. It was nice to manage both of these apps in heroku under one umbrella, but did little other than remind us of the roles of both applications. Next we branched master to a staging branch, and setup heroku auto-deploys for staging via github. Now whenever we push or more likely merge a PR from a feature branch to Staging on github, heroku automatically pulls staging and deploys it to our staging application. This makes doing basic QA before merging to master much easier, it also removes the requirement for someone with permission and git/heroku setup to `git push heroku-staging staging:master`. Our developers can all push or merge a PR, so this made getting work to clients for review much easier and efficient. We now had basic Constant Integration in our staging application.
Iteration 2, one thing that developers still had to do before code reached staging was remember to run `rspec` to make sure changes don’t break, even when this was done, often we’d find a quirk in external environments and tests would fail in production and pass locally because of a strange merge. The solution, adding a CI test runner. Heroku auto-deploy has an option where it won’t deploy a branch until after CI has run. We chose Codeship, and added the heroku addon. After connecting Codeship to our github, it just worked. Now when staging is pushed / merged, Codeship runs our rspec tests, heroku sees the tests pass and deploys our app. We also notify our slack channel but thats another post.
Iteration 3, since all development is being merged to staging, lets add our production app and turn on auto-deployment for master. Now once everything is QA’d in staging, passes CI, and we create a PR from Staging -> Master, if we merge, our master branch will be automatically deployed to our production app. This makes our flow look like (Staging) -branch-> (Feature) -merge-> (Staging) -merge-> (Master)
Iteration 4, staging is too late to code review. We wanted our team to start doing meaningful QA and CodeReviews but it became a hassle. We made an app called koncur which solved the problem of tracking who signed off on a PR, but we were essentially try to do static code analysis and things slipped between the cracks when the code actually ran. One solution was to have all the developers pull the pr to their machine, run it, try to get their db to a state where it was inline with what production environments would be like, all while trying to do their own development. It was awful, and it limited QA to developers (who had to project cloned and up to date). At kohactive we wanted everyone to QA, give early feed back, and describe UX issues. Pulling and running a Rails app for every PR was a non-starter. Enter Review Apps. Review apps are brand new heroku apps that can be generated automatically for every open PR on a github repo. They are amazing but require a little bit of thought and configuration to work properly.
Review Apps. To configure review apps, create / open your pipeline. The first column is Review Apps. Click the chevron next to review apps and enable review apps. Next you will select a parent, this will likely be your staging application. From there heroku will auto-generate an app.json. An app.json if you aren’t familiar is a json file that describes how heroku should build your app. You can set ENV vars, buildpacks, post-deploy scripts, and provision which addons your new app will need. See https://devcenter.heroku.com/articles/app-json-schema. Provided your app.json gets commited and your review apps are on and set to automatically create apps, when you open a new PR a new app will get created, spin up, and deploy your code. Now when you open the review app in your browser it will do one of two things, 1) Not work, “An error occured” or 2) Be completely empty.
Preparing your Review App to be Useful. The reason the app is broken or empty is that review apps don’t get copies of your staging database, honestly you probably wouldn’t want that. A better solution is to seed your new app’s db when the app is created. Seeding a db with rails is very easy. Add or edit db/seeds.rb and put ruby statements that create testing data. If you are using factory girl in your tests, your factories can be used to quickly create test data for QA. To prepare your database automatically on new review app creation, you can add a postdeploy script.
This command will migrate the db up to your latest revision, or you could use `rake db:schema:load`. And then will run your db/seeds.rb
Do your QA. Now that you have a working app and a useful database, get your team to try the code out in the wild. Again, while static code analysis is good for finding obvious bugs, omissions and style conflicts, doing a quick once over of your code before pulling it into your release pipeline will uncover code and UX issues that your CI might have missed.
Follow me on twitter: @j_mcnally
Check my github: http://www.github.com/j-mcnally
My company: https://www.kohactive.com — we build ambitious, rich web and mobile applications that scale.
