The Future is Unknown, Plan Accordingly
Doing new things is not easy, big or small, and there are many ways to prepare. On one end of the spectrum there is a deep analysis to create a big plan and on other side is ready, fire, aim. Both options can be just as effective in delivering. The following two tales are not exactly on either extreme but asymptotically approach each end.
Jumping in Feet First
A while back I worked on a fairly large product that was originally written in C and slowly migrated to C++. The migration to C++ was not quite complete and a significant portion of the code relied upon a custom virtual dispatch system. The reasons for the system was well justified back in the day but had grown unnecessary as computing power had grown and the custom dispatch system prevented a number of conveniences afforded by C++.
Start Swimming
At the time I was quite the whipper snapper and decided that instead of working on a new feature in which I would have to use the custom system, I would complete the migration to C++. I did a few brief tests to feel out the edges of the change and decided the project would take 6 to 9 months to complete and I could do it in parallel with the rest of the team (more than 100 engineers) all working on the same code base.
I don’t remember how I got leadership to sign off on the project because the plan I presented fit on a single page and was a list of about a dozen items that needed to happen in series with minimal off ramps if things went awry. There was some resistance by a few of the senior engineers that felt the change was a bit risky. Needless to say there was enough enthusiasm for the project that it started with various pockets of support.
Watch for Riptides
About half way through the project I ran into a bit of a snag. The original dispatch system afforded a level of flexibility not usually seen or leveraged. A small piece of code in one of the classes changed the type of the object mid way through the loading process . Basically the method in question overwrote the memory of the just instantiated object to point at a different table of function pointers. It went something like
void TypeA::Load()
{
// a bunch of loading code
if (this->_changeType) {
TypeHeader* header = (TypeHeader*)((unsigned char*)this - sizeof(TypeHeader));
header->vtable = &TypeBVTable;
}
// more code
}One of the true benefits and horrors of C/C++ is the ability to cast memory to whatever. I did not expect that to happen and if you had asked me before the project started how I would handle the situation, I would have said we would need to redesign that portion of the system to not rely on the ability to dynamically change the type of an object. Given that I still had months of work to complete I needed a solution now.
Finding a Sandbar
As I stared out the window of my plush half height cubicle in shock I hit upon inspiration. I realized there was a way to overwrite a vtable. You just inlace new the object on top of the old one. As in…
TypeA* a = new TypeA()
TypeB* b = new (a) TypeB()Provided both objects have the same members variables and functions it just works. The two types did have the same member variables since one was a derivation of the other. I tested it out on all the platforms and it did indeed work. BTW, this still works.
I decided to leave things better then I found them and instead of hiding the change in a larger method, I created an appropriately named function. The function was called RecklesslyChangeType. I don’t know if its still there, I hope not.
I was able to complete the project within the yearly release cycle of the product with help from a handful of excellent engineers. I don’t know if I would have started the project if I knew I would have to figure out how to change the type of an object at runtime. Sometimes ignorance is bliss as long as it is followed by hard work and a little luck.
Getting Fix’n to Start
A while later in my career I had the opportunity to lead the creation of a new game development kit (GDK). At the time I had worked on two previous GDKs as both an architect and a team lead. Before that I had worked for 10 years on three different frameworks. Suffice it to say, I had a fair amount of experience building components to be used by other developers.
Planning to Plan
The two previous GDKs I mentioned were still being actively developed and the new one was intended to gradually replace usages of the previous two over the course of a couple years. It was a reasonable plan based on the experience with previous GDKs. A drawback of the plan was that the teams with GDK experience needed to continue to develop new features and a new team would be needed for the third GDK.
The new team would need to be fairly large given the scope of the new GDK and getting head count approval for the project would take a couple of months. The approval process went back and forth for a while to refine the budget and it was also decided that the project would be a good opportunity to try a near shore software contracting firm. The firm would need to be selected before the project could be approved.
Idle Hands are the Devil’s Workshop
I was eager to get started on the new GDK but didn’t have the near shore team lined up. The firm selection was going to take a few months of trips, interviews and discussions. I decided to start planning the project with the input of senior engineers and architects of the existing GDKs while the the review went on in parallel. Collectively me and the existing GDK leads had 20 years of experience building GDKs and I thought it was important to start getting something down in Jira.
By the time the firm was selected and the budget approved my leads and I had a large Jira project with dozens of epics and milestones and hundreds of stories. Not only did we have detailed plan but had I started to socialize the plan with stakeholders. The excitement for the project was building across the organization. Everyone agreed it was a detailed plan.
The Rubber and the Road
After an eternity or more accurately 5 months the near shore company had been selected, the budget approved and the team onboarded with access. I started a series of workshops to walk the new team through the project plan. In a sign of things to come, the sessions did not generate many questions.
The first sprint was started after the workshops and the result was not good. Only a few of the committed stories were completed and the team was confused and struggled to explain the reasons for the outcome. We tried a few more sprints with the same result in both output and retrospective analysis.
I consulted with my leads from the other GDKs teams and leaders from across the company to get different perspectives. Based on the feedback as well as some more time with the new team, I decided we need to start over with the project plan. It was a bridge too far to expect a new team, without domain experience, as English as a second language, to absorb and own a detailed project plan created by domain experts.
The original Jira project was mothballed and a new one created. Instead of trying to build a full plan we started with a small backlog and incrementally built the epics, stories and tasks as sprints progressed together. The team’s understanding of the project improved and they started to deliver on their commitments after few sprints. The team was happier and the stakeholders were starting to build on the deliveries.
Retrospective
In the former tale a minimal plan with a flexible team delivered results in a large project driven by a waterfall process. In the latter, a new project slated to leverage Agile with a small team briefly turned into a waterfall process. Since the last experience I have preferred Agile development with a minimal plan that will unlock budget and satisfy stakeholder concerns. The minimal plan needs to be followed up by regular communication on status and expectations.
Tools like Jira and Asana are good once the team is ready to start putting the plan to paper and executing. I don’t have experience in using either tool to plan the plan and track its evolution during execution. There may be a gap in tooling that captures the forces that shaped the initial plan and then altered its trajectory. I continue to search for a tool to help capture the type of analysis detailed in the tradecraft primer.
