Gardening Day: how to save money, transfer knowledge and reduce risk by removing the weeds
And have people asking for more!
Technical debt slows us all down. Keeping on top of it requires constant work, not unlike maintaining a garden: if we don’t keep on top of it then the weeds can take over.
But no one really gets excited about cleaning up unused Splunk dashboards, organising old AD accounts or rightsizing EC2 instances.
Inspired by this blog post by Håkan Foss and the ft.com team’s documentation day, we ran a ‘Gardening Day’, a single day for a group of teams within FT technology to collaboratively ‘de-weed’ our software and hardware estate.
What did we want to achieve?
We wanted to identify and fix as many different issues within our services as possible, in one working day.
These issues could range in size from small things such as deleting an unused AWS S3 bucket, to much larger tasks like organising our AD groups and identifying and removing those that are no longer required.
Results
By the end of the day we were completely blown away by how successful the event had been. Our teams proved to be quite competitive and minutes after getting started we were receiving our first issues, not long later we started seeing our first closed issues too.
308 issues raised across our services.
270 of these were resolved on the day.
Some of the highlights of the Gardening Day:
- >90GB of NAS storage deleted
- at least $3500/year savings on AWS cost
- 25 unused splunk dashboards deleted
- 450 unused AD accounts deleted
- Tidy up of load balancer pools
- Security improvements by deleting old accounts
- Deleted old network infoblox space to make admin easier
- 3 tubs of celebrations/quality street emptied
Within the FT we have a gamified system called System Operability Score that allows us to rate a systems runbook by how effectively it can be used by our Operations team. We had previously held the top spot within the company but due to other groups improving their resources we had slipped into second place. However through some hard work by a number of our teams, enough improvements were made during the day to bring us back into the lead.
But also
We asked all participants of the Gardening day for feedback. The statistics from this feedback ended up being very positive:
- 100% of participants said they had worked with people they don’t normally work with.
- 90% of participants learnt something new.
- 95% of participants thought we should definitely run the day again.
- 95% of participants scored the day 4+ (out of 5) when asked how useful they thought the day was.
So How were we successful?
A number of decisions we made when planning this really worked out for us!
Start by breaking the ice
We know that in general people aren’t a fan of icebreakers however we found that starting the day with one really helped people mingle. It brought out a competitive side and acted as a springboard for the whole day.
We split the group in two and had them form two lines facing away from each other and then asked the lines to hold hands. One person from the end of each line watched a coin toss. If it was heads, they squeezed the hand of the person next to them once, if it was tails then twice. This was then passed down the line to the other end where the person had one chance to shout out the result of the coin toss. The first team to get the correct answer gained a point, the first team to five points won the game.
Allowing Teams to Self Organise
People were then asked to find their own groups. As participants had an understanding of a range of different aspects of infrastructure and cyber security, we advised people to work with colleagues they wouldn’t work with on a normal day in the office. We wanted to promote knowledge sharing and diversify the problems teams would solve.
This process may have required a bit of facilitating as we had one person working remotely and a few people that didn’t initially find a group. But after a bit of coordinating and nudging, to ensure people mixed themselves up as much as possible, it proved incredibly successful.
Letting Teams Set Their Own Focus
Rather than presenting teams with a pre-decided set of issues for the day we opted to allow teams to decide what they wanted to target. We trusted that any issues people wanted to spend their time putting right would be worthwhile.
To make this as effective as possible, a couple of weeks before the event we announced that everyone should keep an eye out for any issues they wanted to fix during the Gardening Day.
Giving Some Guidelines
Some teams found setting their own focus more difficult than others and so to combat this and reduce the barriers for entry, teams were given 6 different prize categories to aspire towards.
We wanted teams to be able to pick one award to aim for if they were unsure about what they could do to contribute to any of the awards. However, in reality most teams ended up being a good mix of people and teams felt comfortable attempting as many awards as possible.
Keeping the Garden Pruned
There were a small number of raised issues that teams never got around to fixing during the day. To ensure these issues weren’t just swept under the carpet, we created a tool to track them and remind the relevant product owners of them on a regular basis.
Our teams were keen to fix as many issues as possible and so we use these reminders to ensure that we continue to fix these issues regularly going forward, rather than waiting for future events like this one.
Final Thoughts
While we were hopeful that the day would be useful, this was always an experiment. It completely surpassed our expectations. Not only by the sheer volume of ‘gardening’ that got done but also the more difficult to measure, but often as important things. Such as improving learning from collaboration and creating a closer network of colleagues.
There was a real ‘buzz’ during the whole event. Our little corner of the FT is not only better tended, but also an even nicer place to work as a result of just one day.
