Let’s get Dev, Ops and QA talking about ideas
The devops mentality is represented by the much-used venn diagram of developers, QA and operations mindsets converging. This is great and good but how does this actually work?
Let’s take an environment where devs, QA and ops don’t/won’t/can’t talk to each other, the archetypal “over the fence” mentality is rife and an initiative has been started to encourage a devops mindset in the engineering department, this is one of the critical tasks. How do the three groups of dev, QA and ops talk while exchanging their views, their pain points, their ideas? Post-its? It seems somewhat like a nonsense question. Surely a dev can just get up off their chair and go over to the ops or QA area and start a conversation.
Sometimes enabling conversation is not that easy, sometimes politics or dept history gets in the way. But it’s still more than possible, given the right environment, encouragement and methods, and more so, advisable and achievable.
Pain points
The easiest way to initiate a conversation with one of the others in the dev/ops/QA trinity, is to listen. It can be a joy to explain problems in the hope that someone can help and just lending a shoulder to lean on can be a blessing in disguise for someone eager to know where their experience can help.
Operations engineers can have concerns such as stability of the live environment, the ability to monitor the systems running in live so as to get a clear picture when there’s a problem occurring, or about to occur. They could be thinking about logs, visibility, observability of the live systems they support, documentation for those 2am wakeup calls.
Developers can have problems such as reliable test environments, development environments that can easily and quickly be recreated, writing unit tests in order to test their code earlier in the SDLC (Software Delivery Life Cycle) so that they submit less buggy code to the repository. They can have time issues with deadlines, they’ve got to get their code ready for the next release branch being cut.
QA engineers are usually wondering how to hone their test suites for their product, how to get the tests running earlier in the SDLC easier, faster and more reliably, with more visibility on exactly what went wrong and where. The QA engineer interested in devops will be thinking about how to share the responsibility for running tests not just at the integration and production phase, but at the Development phase too.
These problems, focuses and concerns are ideal starting points to have a conversation between tech leads in ops, QA and dev to create a foundation for the understanding of each other. These are often better facilitated from the top-down approach, a CTO or VP of Engineering investigating and mandating communication between the camps but with the right format, it’s fine to be initiated by non-execs.
A problem persists, though, how do we listen effectively, and also provide ideas to try and help with employees’ problems? One person taking the initiative and talking to another person in another group isn’t going to gather too much information about the group as a whole. It’d be great if there was a forum for raising ideas about how things could be made better and allowing collaboration and supporting suggestions, refinements and discussion, wouldn’t it?
Ideas
There are methods to help with this, of course. Open forums in office meeting rooms where the people gather to talk, wiki pages for collaboration etc but one of the best is an open, anonymous-posting-by-choice (but not a requirement), website format where ideas can be voiced and voted on, comments made, where every engineer, every C/VP-level can view the comments and opinions of the engineering teams as a whole. These are often called “Idea Management Systems” or “Ideation” systems.
One problem with ideation systems is that ideas usually mean change and not everyone likes change, especially when it affects them. Some employees are averse to change, an operations engineer might not want to learn git, for example, or get involved in branching methodologies for Github repositories.
Some employees might wish to be completely anonymous in posting up an idea for public consumption that is, perhaps, revolutionary, or containing wide changes that affect people in undesirable (for them) manners. Worse, they might not post them at all for fear of offending someone’s sensibilities that they hold in high esteem, whether or not they feel their idea is worth discussing.
This has an impact on what mechanism is chosen in a company for an ideation system, whether management wants to keep a tight rein on ideas, whether they want to encourage a more open and blameless system with high confidence in anonymity, should the poster choose to remain so, and not put their name to the idea.
There are a variety of mechanisms that ideation systems can follow:
- 3rd party ideation websites
- In-house ideation websites
- Wiki pages/Google Docs/Dropbox etc.
3rd party ideation websites
A plethora of 3rd-party websites have been spun up to service this requirement. As can be expected, these websites will typically charge a fee that changes with the service complexity and elegance but one advantage (apart from not worrying about the infrastructure) is that they are (usually) completely and very visibly anonymous when thinking about employees submitting ideas, as they are essentially a blackbox to employees and staff, ie they don’t see the server logs of the service etc. On some 3rd-party products, there’s no login procedure (Suggestionox just relies on a shared office password, for example, providing no possible identification of the individual employee), management have no ability to view the logs and find the IP address of the laptop submitting the idea.
Staff who might feel uncomfortable raising controversial ideas openly can have a greater confidence in anonymity, when submitting ideas that might not be palatable or that would tread on delicate feelings of pertinent staff. They will probably feel less restrictions in being factual, honest and open (as in a blameless working environment). That said, if an idea is posted that would offend the poster’s colleagues, or possibly impact someone’s working practices, then the idea might still be muted by management out of concern for other employees’ feelings or concerns.
As with the amount of websites out there, there are a range of levels of functionality in these websites. Some will act as truly autonomous forums for ideation where any and all ideas are shown and are up for public discussion, with no (or at least, minimal) filtering by management who would, presumably, be managing the service and the ideas that have been submitted (as, at the very least, they have to block ideas that are offensive that have been submitted for public view by a possibly disgruntled employee).
Other systems will not allow for public visibility or discussion on submitted ideas but simply be a mechanism to pass ideas to the service managers (imagine a service whose aim is just to allow anonymous idea submission to management, for example, where the ideas will be discussed at an executive level).
Examples of these websites, after a quick search for the first few results:
As ever, a minute or two on a search engine will yield heaps of results of services of this type.
In-house ideation websites
There are a variety of systems that are available for in-house ideation websites, typically of varying quality and production-readiness, available on Github.
One example is the, somewhat unmaintained of late but still more than usable, CFPB’s IdeaBox. It’s a system that used to be maintained by the Consumer Financial Protection Bureau in USA and it provides a Python Django website system that allows for idea submission, commenting, up/down-voting and collaboration. It seems well suited for three groups that need a forum for ideas on how to effect a new methodology.
If you’re interested in giving it a spin, here’s a bash script I wrote to install it in a docker container, it takes 5 minutes or so (to be clear, that’s not productionised, it’s just to demo what it can do, but I don’t think it would take too much to get it ready for a Prod environment if you wanted to).
There’s a fair amount of other options, if hosting onsite is an attractive course. A quick search will reveal a list of ideabox/ideation solutions on Github (of varying recent maintenance dates and deployable states). It seems to be a popular subject for training courses so a number of those Github repos are POCs only but again, it probably wouldn’t take too much to get them kitted out for production use.
Wiki pages/Google Docs/Dropbox etc.
In the aim of “Keep It Simple, Stupid”, there are simpler alternatives like a wiki page, Confluence, MediaWiki, Wikia or the like. These will allow a page to be publicly viewed, updated, commented on and occasionally have voting plugins like this plugin for Mediawiki, or this plugin for Confluence.
This is a convenient mechanism to store ideas, but if anonymity is a concern, it will rarely provide this as page histories are normally kept along with logins associated with changes. Additionally, wikis don’t typically allow for more than one page editor at one time, which can impede progress and discourage contributions.
Dropbox will follow the same kind of problem where a file is edited locally and synced up to Dropbox. Google Docs is a bit more elegant in it’s ability to allow editors to synchronously edit the same document or spreadsheet but can get a little confusing due to it’s “everything on one page format” if multiple editors are placing responses on the page.
All these systems rely on a basic flat document, though, whether it’s a page or a spreadsheet etc, so it might be hard to manage the ideas, comments and votes in a coherent manner.
It is, however, the easiest and quickest way to get something off the ground, to gain traction as a POC (Proof Of Concept) and if desired, a more elegant solution can be considered down the line.
Idea tracking and collection doesn’t need a glossy, pretty website, a forum just has to exist and colleagues will usually populate it with a mass of problems and responding suggestions.
Things to think about
All said and done, outside of the technical piece, the simple art of managing ideas and creativity is not the easiest thing in the world. The imagination of devs, ops and QAs who suddenly have a voice and a platform for their idea can be an intoxicating experience and some of their more outlandish ideas can make it in to whatever forum is in use. This can occasionally obscure more realistic ideas that would have more chance of take-up and cause users to lose faith in the effectiveness of the platform so careful pruning and education of employees might be required.
Additionally, in all these service forms, business management will usually require the right to be able to block or “handle” any idea offline that’s offering to incite riot in the poster’s colleagues due to being a bit too idealistic or change-inducing. That’s just a necessary evil and should be accepted in most companies as a prerequisite.
That said, providing a capable platform for ideas to be suggested, discussed and eventually chosen or discarded as a pool of concepts for future workstreams to draw from is of massive value to any engineering dept (and, to be honest, any company as a whole, not just the engineering dept).
Any kind of open, safe arena for communication can only be a benefit to three groups who historically tend to not talk to each other much.
Choose a project or two for a POC, get them running and sort out a Powerpoint and demo for your engineering team. It’s worth it to get the people who can make the devops process work talking and you might be surprised at what comes out of it.
