Introducing Cubano: A story of evolution using Living Documentation!

Jimmy Kemp
Sep 3, 2018 · 8 min read

Cubano is a test automation framework written in Java that provides a structure for developing acceptance and regression tests so your team can hit the ground running and not have to waste time needlessly building and maintaining your own framework.

To see Cubano working in action and get some ideas of the types of features Cubano contains, take a look at the Cubano Demo project.

Once you are ready to start writing your own tests, pull down the Cubano Template project and adapt this base to start automating your project.

While Cubano can be used to write standard JUnit and TestNG tests, the real power comes when you use it’s integration with Concordion to create beautiful living documentation that can drive the system under test — whether that system runs in a browser, service or device.

Cubano

  • As a package of quality open-source projects, Cubano includes everything you need to build tests and create beautiful documentation for your Web and API projects.
  • Is a “ready-made” test automation framework that provides everything at your fingertips. It’s ideal for software delivery teams who want to collaborate around Living Documentation that is applicable to a wide audience.
  • As part of the Concordion organisation on Github, it includes Concordion, Selenium WebDriver and other open-source projects. It is written in the Java programming language and requires your tests to be written in Java or Groovy.
  • Co-ordinates and manages Concordion extensions, as well as managing the setup of Web (e.g. Selenium WebDriver) and API Tests.

Living Documentation

Hold on … Living what?

The team at Neuri Consulting in this article do a nice job of describing Living Documentation. The text below is an excerpt taken from that article.

Living documentation is a reliable and authoritative source of information on system functionality, which anyone can easily access. It is as reliable as the code, but much easier to read and understand. Support staff can use it to find out what the system does and why. Developers can use it as a target for development. Testers can use it for testing. Business analysts can use it as a starting point when analyzing the impact of a requested change of functionality. And it gives us regression testing for free.

But still, what does it actually look like, and how can software delivery teams collaborate around it?

With the specifications written in plain language, and styled using Concordion, the generated living documentation is readable by anyone.

One of the key items to recognise is that the screenshots are of a real system under test. So the output is ‘Living Documentation’; no more static documentation.

The storyboard feature allows users to easily see the steps that are performed to implement the specification, and has proven useful for product owners, analysts and training staff. The Storyboard under:

expands to show:

The Storyboard is also useful for showing the contents of API messages:

Finally, the living documentation includes log messages (with embedded screenshots and API messages) that are useful for developers and test developers:

The examples above also highlight Cubano’s co-ordination of Concordion extensions, for instance:

The Concordion website has many more examples.

Back to the start: What’s in a name

The original authors and current maintainers live in the best little city in the world > Wellington, New Zealand. Cubano derives its name from Cuba Street, one of the most prominent streets in our city.

Cubano’s Origins

Cubano’s original code base can be tracked back to two key projects at Te Manatū Whakahiato Ora, the Ministry of Social Development (MSD). Thanks to MSD for contributing this framework to the open source community, and specifically to the Concordion project.

Test frameworks are a dime a dozen?

There are loads of frameworks out there of course; ones you pay for, others that are free and some that have a hybrid model. You can find them by initiating an online search (e.g. ‘test automation frameworks’), or even searching through Github projects. I’m sure as soon as we publish this article, someone will send us a link to one!

Nevertheless (and history is vague as I am sure you can imagine), the orginal test automation suite started out with Concordion and grew from there.

Initially there was quite a bit of moving and copying of class files between these two projects. This process was not ideal of course, but sometimes you need to manage priorities accordingly. And at the time, developing a ‘test automation framework’, as a shared library, was not the top priority.

Eventually however, forced in part by the departure of key team members, the initial version of Cubano was born — the ‘Test Automation Framework’!

We wondered at the time, partly ironically:

”Who really needs another ‘Test Automation Framework’?”

So why open source the framework?

Two initial benefits could be realised:

  • The addition of this framework to the Concordion family. Concordion did not have a framework that co-ordinated the core project and extensions into one cohesive project
  • A powerful way for MSD to continue to derive benefits from this product, without having to maintain it themselves

As a reward for this hard work, Cubano was recognised in the 2018 New Zealand Open Source Awards (Open Source in Government).

Adoption across MSD

Cubano continues to be adopted across MSD, primarily for the benefits derived through Living Documentation.

Cubano is used on both fully integrated systems, as well as mocked or stubbed systems. The targeting of tests for these differing system types (i.e. integrated vs mocked), is of course different, so it speaks to the range of options available through Cubano.

A recent project has used Cubano to compare JSON data sets and deliver the differences to end users. Making extensive use of the Storyboard extension.

It’s not a Panacea

This framework is not a Panacea, and does not perform ‘Automagic’ (thanks Jim Hazen). Cubano cannot on its own deliver outstanding results. Outstanding results are derived through outstanding inputs, trial and error, experiments and continuous improvements.

Lets digress for a moment to make the point.

Craft Beer

Here in Wellington, we really enjoy our craft beers. We’ve got craft beer bars, breweries and bottle stores. These folks really know their stuff, and produce some fabulous products.

Now I’ve never brewed beer before, and likely if I did, my first batch would be pretty ordinary at best. But I’d learn a great deal from the experience, that I could apply to my next batch.

Just like the craft beer brewers, we should treat our profession as a craft, constantly trying, experimenting and improving our inputs and outputs.

A couple of anecdotes to support these thoughts.

Aeons ago during a lecture at Otago University, we were discussing ‘Practice makes perfect’. The lecturer offered this view, often credited to the legendary football coach Vince Lombardi, “Practice does not make perfect. Only perfect practice makes perfect.”

And Albert Einstein is broadly credited with exclaiming “The definition of insanity is doing the same thing over and over again, but expecting different results”.

Read articles from the likes of John Ferguson Smart, Angie Jones and Bas Dijkstra, among others. We’ve used their ideas (and many of our own), in our test suites. This has resulted in working code that complements the theory.

Practice widely and with as much perfection as you can muster.

Further, proficiency in a programming language, is a great input to achieving outstanding outputs.

To that end, because Cubano is built using Java, proficiency in Java of course helps a great deal. Richard Bradshaw’s free online courses for example, are a fantastic way to get started.

Outstanding Inputs

We have a team wiki with guidelines on our test automation approach. This wiki is jam packed with thought provoking articles that we’ve found on our journey.

There are lots of thought leaders out there who contribute to the community. I bet many more that don’t make lists like these. Do some reading and research, and find a style or voice that complements your own.

We undertake code reviews, both manual and static, we refactor often and all of our builds are in Jenkins. We are judicious about what we can test, what we cannot and what we shouldn’t. And we’ve also been incredibly lucky to have the benefit of highly skilled software developers working on our team. This has enabled development of software patterns that have made our code more robust, extensible, maintainable and less susceptible to tests failing due to a framework issue.

We’ve made loads of mistakes along the way; but we’ve learnt from our experiences so that we are able to articulate the merits of what we can test, what we cannot and what we shouldn’t. As Jim would say, it’s not Automagic.

Finally, the Concordion website contains some fabulous material. A particular page I refer to often is the Hints and Tips page. When I am writing tests, I use this page as a checklist to help guide me to writing better software.

Enjoy!

How can I use Cubano?

To really benefit, put down your tools and have your team discuss examples for upcoming features. Get input from a range of disciplines. Use a technique such as Example Mapping or run Specification Workshops.

Once you have examples and are ready to automate them, take a look at the Cubano Demo project for the types of features available. Then pull down the Cubano Template project and adapt this base to start automating your project.

Complement the examples in these projects, with a read of:

Contributors

Concordion

News and updates from the Concordion Project

Jimmy Kemp

Written by

Test Automation Manager/Coach/Practitioner

Concordion

News and updates from the Concordion Project

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade