Tools for achieving devOps goals
DevOps as a concept can be confusing, especially for engineers like me who have not spent much time in operations. Devops has been referred to teams, practices, tools, policies and even culture. Devops as a term started with culture and slowly moved towards being more tactical. This post is about the tools that can be called ‘devOps tools’ as they can be used by teams to achieve their devops goals.
Although ops maturity will not come to teams just by picking the right tools, understanding them is a good first step in that direction. Do these tools make sense to you and your team, will be determined by your goals and the constraints you have. I have tried to enlist the tools and their usage here which worked for my team.
CloudWatch Metrics and Alerts: Monitoring infrastructure (inc. servers, databases) and even Applications can be hooked into Cloudwatch for metrics. NewRelic is also good and provides much better Application Monitoring services but has much higher pricepoint.
ELK (ElasticSearch, Logstash, Kibana) stack: Storing fresh logs (usually few weeks) for search, filter and exploration. Old logs can be pushed to cheaper storage accessible for auditing. One problem with this is aggregations cannot be trusted since Elastic Search makes assumptions while giving you estimate of your aggregations which may not always be accurate.
Graffana: Pretty dashboards for metrics.
Jenkins: Automation (especially build automation), running manual jobs which are not yet automated completely. Has good integration with github triggers and spinnaker. Since it is self hosted, it can run within your Virtual Private Cloud.
Spinnaker: Many developers confuse spinnaker’s usefulness with Jenkins and considers it as an alternative with slightly nuanced features. However, both play along nicely together and have different strengths. It gives you out of the box features for blue/greed deployments, baking AMIs (Amazon machine Image), cluster resizing, updating instance type / properties, rollbacks. With some thoughtful planning it can also do database migrations deployments as pre-requisite to code deployments depending on your workflow. What it does really well is breaks down a big jenkins pipeline into small, modular stages which can be independently managed, rollbacked etc. Ways of spinnaker forces you into the ‘immutable infrastructure’ mental model. You can either create infrastructure pieces or destroy them, cant modify. This ->blogpost is a good starting point for Spinnaker. One big problem with spinnaker is the configurations live in spinnaker and not in code or a repository so they dont really conform with the git workflow.
CloudFormation/Sceptre: CF is infrastructure as code, lives in your version control. Sceptre allows you to pass variables and parameters to CF according to environment (dev/staging/prod etc) and other dimensions. Sceptre is what truly makes CF flexible and usable across all stack without any other complicated mechanisms.
Trello/Jira: Planning, breaking down tasks is vital for achieving devops goals. Use any of your favourite method for project management here as well. Also these tools keep rest of the team abreast with the status of tasks.
Make: As a very new make convert, I have to add Make to a good ‘devops’ toolkit. If you are still on the other side and wondering why even use Make over ‘bash/<insert any alternative>’, I strongly suggest you just try it for one project and you would start wondering why ever use anything else.
Ansible: Standard configuration management, much like Chef, Puppet, and Salt. I prefer it just because it is written in python and its easy to dive into its codebase anytime needed. I dont prefer to use it as a deployment-tool because modifying servers can be a big trap. It is just a preference for me, not necessarily a gospel of truth. In case situations demand, ansible can act as a very good tool for it too.
If you have reached till the end, that means you are 1. Interested in Devops and 2. New to it. Welcome to the world of devops practice, it is going to be an exciting journey. You can impact hugely to your organization. The tools listed here are just for the start, pick and choose them according to your needs. Devops is not about these tools but they are very valuable in reaching where you want to go. They are neither necessary nor sufficient for a good devops practice, anyone telling otherwise is lying.
