Using Open Project for Sprint Planning and Estimation and Reporting With AI

13 min readMay 27, 2024

This article is about how we been utilizing Open Project for managing our sprints for game development. Primary target audience for this article is

Managers
Tech-leads

I’m not going to go into why we use open project. There are other tools you can use them if you prefer. It doesn’t really matter which tool you use to manage your project as long as you can utilize it to fit your needs.

Our CEO’s 1st and foremost requirement for any project is that it needs to be planned. By planning he recommends that we do scrum. Basically, list out what we need to do and estimate how long it needs to be done.

It’s easier said than done. Most often we really have no idea what we would be doing. And it’s very difficult to estimate how long something will take when we have a lot of unknown factors. And if you ask me that’s exactly why it needs to be planned. Because once you start planning you would be asking the right questions that clear out the gray areas. Potential risks and blockers and dependencies. Once you start working in projects where multiple people are engaged it’d be close to impossible to avoid blocking or dependent tasks. And you definitely need to know how long the project might take. It’s not like you can run something indefinitely. Because every project has a budget. For any software/game project there are two primary currencies

Time
Money

Without a proper plan you can’t allocate/utilize them properly. And why this is necessary? Opportunity Cost. I’m not going to go into detail about what opportunity cost is. But with this you can decide which project to invest your time and money in.

Anyhow let's get back to how we been managing this. I’m only sharing the current state of our process. It wasn’t like this when we started, and it won’t be like this when we end. Each iteration only makes it better.

Sprint Duration

It’s not like we invented these. These are already in practice even before I was born. Ideally a sprint duration is about 1–4 weeks. For our case we found that a 3-week sprint followed by a 1-week sprint fits our team workflow better. It could vary from project to project. The reason we do this is because to respect the 80–20 rule. What is the 80–20 rule?

80% of the work can be done in 20% time. And the rest 20% work needs 80% time.

The 3-week sprint is focused on clearing out User Stories as fast as possible. And when we go as fast as possible, we most often create technical debts. While there are mechanisms set in place to minimize the technical debt in the project it’s not 100% possible to avoid it. So once our 3-week sprint is over we get an artifact. A game build that has new features which can be tested. Now we can cross examine each other's work and fix potential risks that might blow-up in the face later. We don’t over engineer. Spending 80% time to get the 20% output isn’t exactly that valuable when we start the project. We need that 80% visible output and while we do that, we make sure that we don’t leave a time-bomb behind that will potentially harm us. Hence the 3-week sprint followed up by a 1–week cleanup sprint. Essentially, it’s a 4–week sprint which has 2 parts. Develop & Polish.

Planning & Estimation

It’s very important to understand that estimations are done in multiple levels. We can’t just go and estimate everything in a project. It may be possible for projects that are small and maybe have a duration of 1–3 months. What if it’s going to last years? What about maintenance projects that has no time restrictions? That’s why we estimate in multiple levels.

Since we are talking about sprint planning, we would focus on estimations that start from this level.

Each sprint starts with a goal. The goal can be a checklist of the outputs that will be visible at the end of the sprint. Once we have set the goal we break it down in User Stories. In the sprint meeting we only plan and estimate with User Stories. Usually, each sprint meeting takes 4 hours to be planned.

In this we discuss potential blockers and dependencies. These are called sprint impediments. While keeping these in minds we define the User Stories. And their Acceptance Criteria's. What are acceptance criteria? Basically, an agreed checklist that mentions what are the states that will be checked before we accept the User Story as complete.

It doesn’t always have to be a checklist. It can just be a paragraph.

Even though we been talking about estimation. We do not do hourly estimation on User Stories. Instead, we estimate with Story Points. What are Story Points? You can check this video here for a better idea.

But for simplicity let's just say 1 Story Point = 1 Developer Day worth of work. So, for us 1 Story Point is 8 hours' worth of work. Why we do it? Check the video. Or google it up. This is out of the scope of this discussion.

Anyhow we put estimates in story points for each User Story. And when we do make the estimation, everyone is there. It’s cross-examined and vetted. And we also discuss how we are planning to do what. We assign who’s going to take care of which User Story.

Thanks to this now we can clearly see who’s assigned how much work.

We can see if someone is overburdened with tasks
We can see if someone has bandwidth to do other tasks.

This way we can ensure a proper work distribution. So how we can do that? So, let's say the duration of a sprint is

3 Weeks = 15 workdays

Since each story point is worth 1 workday, we can safely assume that each assignee should have 15 Story point worth of work assigned. Anything less than that means proper utilization is not there. Anything more means they are overburdened. It’s very important to distribute work properly.

This is how it looks when the meeting is done.

Daily Standup

While we surely talk about the tasks that need to be done for each user story, we don’t make hourly estimation for each task that day. Insead what we do is. From the next day we have daily standup. In the morning everyone sets 10 min of time for this meeting. This meeting is very simple. Everyone goes one by one, and Say’s I did this yesterday and I’ll do this and that today. And will take x hours for this task and y hours for that task. The project manager takes notes of those tasks, and the hour estimates in an excel sheet. Immediately after the meeting, The PM Goes to our Open Project instance and opens up the Sprint Task Borad. It looks like this

The board already have the User Stories. He goes ahead and creates tasks under each story and puts work estimations that he received from daily standup.

Now each Assignee gets their own assigned task in the Open Project Platform.

At the end of the day, they log the actual spent hours needed to complete the task. Sometimes it takes longer than the original estimated hours, sometimes it takes less than the estimated hours. It’s perfectly normal. The more experienced you are with estimation the more accurate the estimated hours vs spent hours would be.

Since all task is under a user story. the estimated hours and spent hours gradually get summed up in there. And finally, we get to see this.

Fig: Story point vs Work Estimate vs Spent Time

Analysis

1 Story Point = 8 hours

Let's analyze some data from the screenshot on top.

#568 Library Room Interior

+-----+------------------------+----------+----------------+-------------+------------------+----------------+-------------+------------+
| ID  |        Subject         | Category |    Assignee    | Story Points|       Work       | Remaining work | Spent time  | % Complete |
+-----+------------------------+----------+----------------+-------------+------------------+----------------+-------------+------------+
| 568 | Library Room Interior  |   Art    | Iftekhar Alam  |      10     | · Σ 75.0 h       |                |     71.0    | 0% · Σ 100%|
+-----+------------------------+----------+----------------+-------------+------------------+----------------+-------------+------------+

From the report we can understand that we had User Story called Library Room Interior which is complete. Originally in Sprint planning meeting we had estimated 10 Story points. Which translate to 80 Hours. But when we actually did the task, our estimation was 75 hours. Which is very close to 80 hours. And Actual spent time was even less than those 71 hours. The accuracy of Story Point Estimation vs Actual Spent in this Case is 112%.

From this information we can come to some conclusions.

The User Story was complete within the budgeted time. And it took less than 7 hours (1 day) less than the original estimation. So, we are now ahead of Schedule.
The User Story was very slightly Overestimated 12%. Which is withing acceptable range (+-20%)
We are good at making Estimations for art categories or Iftekhar (assignee)

Let's take Another One where we didn’t do as good.

#717 Comic Style Cutscene Package

+-----+----------------------------+----------+----------------+-------------+------------------+----------------+-------------+------------+
| ID  |          Subject           | Category |    Assignee    | Story Points|       Work       | Remaining work | Spent time  | % Complete |
+-----+----------------------------+----------+----------------+-------------+------------------+----------------+-------------+------------+
| 717 | Comic Style Cutscene Package|   Code   | Zubair Islam   |      3      | · Σ 46.0 h       |                |     50.0    | 0% · Σ 100%|
+-----+----------------------------+----------+----------------+-------------+------------------+----------------+-------------+------------+

From this we can get the following analysis

This task was heavily underestimated. Story point was 3. But the Work Estimate came up 46h. It was supposed to be something around 3x8= 24h.
Spent time was 50h which is more than the task estimation 46h. The accuracy of this User Story was 48%. Which is 52% off the mark.
So, we should be more careful when we are to do similar task estimations.
We are now behind the schedule.
We should do a proper retrospect of this user story to understand what we did wrong and how can we improve our estimations in the future. Something like this

Now if you are Lazy. You can generate similar reports with ChatGPT. You can simply paste the raw data to the chat. And after a few guiding prompts you can generate something like this.

However, be very careful from just pasting reports from ChatGPT. It’s a language model. It will make mistakes and make things up. So do read them before using them

Thanks to this we can now take effective decisions on where we need to improve. And by exporting data from open project, we can make further analysis for each task.

Trouble in Paradise

Let's talk about some common problems that you would surely face when managing a project.

What if sprint ends but a user story isn’t complete.

This shouldn’t happen. Because you know the duration of the sprint. And in sprint planning you shouldn’t have any user story that has more story points than the duration of the sprint. What it means actually is, you didn’t break it down enough. What you are calling a user story is actually an Epic. We didn’t go into details of Epic. Ideally the Project’s estimation would be based on Epics. Epics would have a list of User Stories. And User Stories will have list of tasks, bugs etc. So, if your User Story is continuously being dragged from sprint to sprint this means it’s very likely not a User Story. It’s an epic. And you should break it down further while keeping it as a parent.

In short at the end of each sprint all User Stories must be closed. If you have Story that can’t be closed or it has more story point than the duration of the sprint, convert it to an epic and break it down further.

Why do daily standup? I have the same task every day.

This shouldn’t happen. This is a clear symptom of tasks not being broken down properly. If you are doing the same task every day, it’s not a task it’s probably a User Story. Convert it as a User Story and break it down to tasks. No task should have more than 8 hours of estimate. Always break it down to lower than that. Ideally no task should ever be more than 4 hours. Anything more should be broken down into smaller chunks. So, if you have this symptom breakdown your tasks properly.

I have an R&D Task. I don’t know how long it’s going to take.

Lets remind ourselves why we estimate? We have a project deadline, right? No project has infinitely long-time frame. Why are we doing R&D? Because there’s a gray area that we are unsure of. For me I have two steps of doing an R&D Estimation. First thing Timebox it. If you have no idea how long, it’s going to take it’s clear that you didn’t do any research. So, allocate time for the research. Depending on the criticality of the feature you need to allocate a time that you will do research form. And after that time you have more information a better estimation of how to do the development. So basically in R&D we first need to have a time boxed research time. After that we can do a better estimation of Development.

Now I know what you are thinking. What if the research output comes out negative? I mean whatever we were researching results in failure?
Simple. With the new found knowledge either discard the task as it’s impossible or timebox another Research Period that might yield success. It’s not like you are an oracle. However, It’s important that you timebox your research period. Since you don’t know how long it’s going to take start small. Allocate 1 hour for that. After an hour you should realize how much more time you need, and it’d be a better estimate now.

So how do we put this in our sprint?

Ask yourselves questions. Most often you need to show your research output by the end of the sprint. So if you have a R&D Story. It can’t be more than the sprint duration. If you know it’s longer than the sprint duration, then maybe break the research part for this sprint and do the development in the next. If your research requires more than one sprint. Break it down even more. However, if you know this much how are you claiming that you don’t know how long it’s going to take?

In any case, don’t be lazy. Time box it.

How to setup acceptance criteria for RnD projects

Who’s the stakeholder? You? Your Boss?Client? Whoever it is, it’s their role to verify the acceptance criteria. A very important reminder. Acceptance criteria is there to protect you. Not your clients. So, make a checklist of the deliverables. While it’s something that needs to come from them, you need to write it and get it approved. You need a clear approval or sign off that these are indeed the list of things that needs to be done for this task to be accepted. Remove a few items from the list before you sumbit. Why? Because it’s very important that you let your client/boss to give you some tips. Because they would anyway. Even though you probably know it. It’s just better to leave obvious blanks that they can fill so that it’s totally within your expectation. If you do not do so. They would tap into their creativity and pull new tasks out of thin air that you would be probably hearing that for the first time.

With that you should have a signed checklist. And yes of course when the time comes. They wouldn’t accept the checklist anyway. They would want to add more things. That’s actually up to you how you handle. New task new sprint can be a great way. However ethically and legally you do have a signed/approved checklist . And legally it should be okay to close the ticket if that checklist is complete. And you should be able to push them to accept.

No, I was asking how to like to write the checklist?

Did you really sit down to write the checklist? Yes? Are you sure you really understand the requirements? No? Okay at least list up the requirements and copy past them to ChatGPT and ask it to write a list of acceptance criteria for these. Now it should help.

If QA rejects any tasks, how to handle that?

You didn’t meet acceptance criteria. If this is the case, why did you submit? You too did agree to this acceptance criteria. Go back and finish them.
You did meet the criteria. However, Requirements changed, or new expectations added. Sounds like a change request to me. Do you have a written agreement on how change requests would be handled? No? Write it. Ideally this means a new ticked. Push this to be accepted and a create a new ticked with new expectations. At least that’s what you should get legally.

You keep mentioning legally and ethically. Why?

If you start to resort to legally and ethically parts, you probably need to start rethinking who you are working for. Why are you doing project management if you aren’t going to follow it?