Estimations & Expectations
Last Friday, we had an interesting session about estimations. Part of the apprenticeship at 8th Light is to learn how to accurately estimate how long it may take you to complete a piece of work, and the difficulty of the task becomes apparent very quickly when you need to apply it to your own work.
Why we estimate?
We’ve all fallen prey to Parkinson’s Law where work becomes gaseous in a box made of time and “work expands so as to fill the time available for its completion”. If you are given three weeks to complete a piece of work you could complete in two weeks, it will likely still take you three weeks because you “have time” and you “will do it later”.
Estimations allow you to give yourself time limits so that you do complete tasks. The goal doesn’t just become “completing the task”, but “making sure it’s complete by the given date”. This marker becomes useful to keep us focused, because the task becomes finite and we are not so daunted by the prospect of it never ending.
Estimations are also crucial in professional environments because we don’t just work on our own and for ourselves anymore. The work we do is part of a bigger picture, and other people and other projects may rely upon what we produce. In this case, the estimation doesn’t just help us, but the rest of the team as well. It will help them work out when the next part of a project can start, if a product is ready to be sold or released, and also how we work, but we’ll get back to that last point later.
What we estimate
During the session, my mentor shared with us this repository, in which he gave pointers as to how to estimate things, but most importantly what can be estimated with regards to engineering work.
This was a crucial point for me because I am still very shaky on my estimations. I tend to forget the kind of issues I inevitably end up encountering (how long it takes to debug, for example, or writing tests if I haven’t TDDed my work). The list provided, and which I have partially reproduced here, is a good reminder of what I should bear in mind when I estimate work.
- Documentation
- Integration to the current system
- Implementation of the code
- Code review
- Testing (unit, integration, manual E2E)
Recently, I reached a point in my work on my Tic Tac Toe game where I had all the functions I believed I needed to put together my game, and they had all been tested. I figured the hard part was done and I’d have a working game within the day.
How wrong was I. I had painstakingly cut out a bunch of puzzle pieces, but when it came to trying to put them together to see the end picture, they were all wonky and didn’t fit where I thought they should fit. In the end, it took me another two days to fine-tune my pieces, and even then, the edges are still very rough!
I had failed to anticipate that I might need to rework some of my functions so that they could work well with each other. I had also failed to anticipate how to debug in Clojure, and how long it would take me to make sense of the error messages and the stack traces (there’s a great article about it here). I had also failed to anticipate having 14 of my 18 tests failing as a result of the changes I had to make, and how long I had to work to fix those.
Development is more than just writing the code, and the ecosystem around the code you produce is often forgotten in estimations, which can lead to wild discrepancies in the estimations you give and how long it ends up really taking you to complete. This ends up being demoralising to you, as you feel less productive than you thought you should be, you might even think you’re a worse developer than you are.
Sadly, it also gives a poor reflection of your abilities to management, who might think you inefficient or lazy; the rest of the dev team who might have been relying upon your piece of work to deliver another piece of work; and other team members who may not have as good an insight as to how tricky estimations can be when dealing with code and what you’re actually faced with (we all know how fun it is to debug work that relies on external libraries…)
How to estimate
So how can we be better at estimating? Ultimately, I think it is a game of trade-offs. Some I’m aware of, some I’m still learning about.
Is it better to have an incomplete product that is well-constructed comes release day, or is it more acceptable to have a complete but rough-around-the-edges product that people can use, even if they find bugs?
Is it judicious to spend time cleaning up your git commit hours before a deadline instead of fixing the final error that’s stopping you from ticking all the boxes on your ticket? (Hint: not really.)
These decisions are your own call, and should be made in accordance with the context you evolve in, the team you work with and the project you work on.
Estimating is a tricky beast because of all the things that need to happen on top of writing code, but also because our days are not made of eight consecutive hours of coding, and in fact, we are interrupted too frequently to make that happen. So how can we mitigate for that?
The list I linked above is definitely a step in the right direction if — like me — you’re not the best at remembering that a ticket isn’t just the code, but also the code review, and the debugging and the integrating to the current system. Bearing these items in mind while you work, you can conduct data analysis on your own work to figure out what are the lengthier steps for you and make your future estimations more precise.
Another concept that I’m still wrapping my head around is time-management. Not so much in terms of what I work on during the day and for how long, but rather in terms of the discrepancies that exist between your estimation of how long a piece of work will take you, and how to reconcile that with your actual work day, that is likely peppered with meetings, stand ups, retrospectives and other non-codey things.
Are these things that I should take into account when I estimate? Should I plan for a day of work for a ticket I will likely complete in a couple of hours because I have meetings to attend? Should the given estimation solely reflect how long I believe it will take me to work on a ticket? If so, how do I communicate the status of a project when I estimated it to be a day but I’m still working on it a week after starting it?
Thankfully, I don’t have to reinvent the wheel as these discussions have certainly taken place before I even thought about becoming a developer, and here are a few resources I have found useful to get a better grasp of different estimating techniques I could use and how to better understand what goes into software estimation:
- Evidence-Based Scheduling — blog post by Joel on Software
- The Cone of Uncertainty — Steve McConnell
- Software Estimation: Demystifying the Black Art — Steve McConnell (Safari link)
- Agile Estimating and Planning — Mike Cohn (Safari link)
- How to Get Better at Estimating Software Development Time — Hackernoon
Estimations VS Expectations
I don’t think it would be fair to talk about estimations without talking about expectations. After all, we share estimates so as to manage expectations. The problem is that expectations can become harmful when estimations go wrong.
As I mentioned earlier, estimations are partial reflections of how we work, and the more accurate the estimation, the better reflection it will be. Estimations do not tell people how good you are at what you do, but they can be perceived as such. If you constantly take longer than expected, it may not be because you’re a terrible programmer, but rather an optimistic one.
You assume a piece of work can be done quickly, forgetting that it needs testing, documenting, checking that no libraries have been updated in the meantime, etc. You will have completed the implementation of your code in the time you allotted yourself, but your failure to anticipate all the other bits and bobs has pushed your deadline back.
No matter how beautiful, clean and well-designed your code is, product might complain because the work is late, or your manager will assume that you work slowly or inefficiently because your velocity does not necessarily take code design into account, and you will end up having a bad time during your next review.
Estimations, ironically, can also end up impacting your self-esteem. Constant mis-estimation may leave you to feel useless, inefficient, incapable or working too slowly, which in turns impact your work in a detrimental manner, and this all becomes a self-fulfilling prophecy. You end up giving “pessimistic” estimations to avoid being late again. You feel like you can work faster than what you said you could, and try to underpromise to overdeliver.
But what if that “pessimistic” estimation was just your ego talking? What if actually, this wasn’t a “conservative” or a “pessimistic” estimation at all, but an accurate one? What if this was your optimal pace of work? Would that be so bad? If we are by nature optimistic about how fast we can deliver work, I also believe we want to project an image of ourselves that is better, faster, more productive that who we really can be.
The beauty of this, however, is that we are all like that, and all in this together. Of course we want to make ourselves look good, so we overpromise. We tell people we can complete work faster than we truly can, because we think we’ll follow the happy path to perfect code that does not need to be amended after a code review or manual testing. This does not make us bad developers, but it means that we may need an injection of humility.
Of course, there are pessimistic and conservative estimates, but these should not be set solely based on your own work, but rather on other variables too, like the roadmap of the product, the requirements of the business or your team, etc. These variables however, should also be taken with a pinch of salt. The roadmap of a startup might be way too optimistic, and if your estimates are deemed too slow or conservatives, it could also just be that the team needs to readjust their viewpoint — it’s all relative!
But in order to keep a sustainable pace in your work without giving up your work-life balance or your sanity, it could be time to become more conservative and see where that leads you to. In all likelihood, your co-workers will see you as a more dependable and reliable worker who can be counted on to deliver on time. After all, it’s not the speed at which you work that truly matters here (at least not always) — code will be done when it’s done — but how your communicate with other team members.
Being true to your estimates means that others know what to expect of you, and are less likely to find themselves in tough spots because they promised a feature would be delivered two weeks ago and you’re still waiting to have it tested after discovering that a module you used had disappeared (left-pad, anyone?). Finding your optimal pace and sticking to that data you previously collected when estimating will also mean that you will not be under so much pressure to achieve your goals. Ultimately, without having to worry about whether or not you’ll make the deadline, you can code more serenely and produce higher-quality code, which will serve everyone better in the future.
