The Fog of Development


Fog of war is a phrase used to summarize uncertainty in armed conflict. Tactical plans and combat strategies, no matter how carefully crafted, have only a limited degree of certainty. This limit of certainty is governed by limits of information, and is precisely the same factor that limits a developer’s ability to accurately or precisely estimate tasks.

This is the fog of development.


“Just reorder the posts in that list to show the newest ones first”

The above task seems innocuous enough. If you’re doing that with a SQL query, it’s as simple as an order by clause. If you’re doing that in a CMS, it’s as simple as selecting the right option from some dropdown. Right? No problem! 30 seconds, tops. I got this.

At least, I thought I got this, until it took me the better part of 2 hours.

I was working on a Joomla project whereby a K2 category page was showing a list of items in alphabetical order, instead of chronological order, with newest first. K2 is a content creation kit for Joomla that acts as a drop-in replacement for Joomla’s limited content creation abilities.

My first instinct was to go to the K2 category that governed the list of items being shown on the page in question, and set the “item ordering” drop down located in an obscure place in K2's accordion menu, to the appropriate value. Upon arrival, I saw that the ordering was indeed set to alphabetical, said “ah ha!” like a cocky bastard, and changed it to “most recent first (by date created)”. Boom. Done. Right? Nope.

After the most obvious fix failed, I wasn’t discouraged, I still had a few tricks up my sleeve. I knew caching was disabled for development, so the next thing I tried was adjusting the same settings for the menu assignment for that category, and that’s where I encountered my first problem — which menu? There were approximately 16 menus for this project, and Joomla does not provide a way of searching all of them. So I had to go through one by one to find the menu this category happened to be associated with. I got lucky and found it on the second or third try, but time lost is time lost. I’m now well beyond my 30 second estimate.

Anyway, the first thing I see on the left is a dropdown called “Ordering”, whose value is currently set to “ordering” (because that makes sense…). When I click the dropdown to see my options, I’m not presented with a familiar list of choices like “most recent first” or “last modified”. Instead, I see a list of …. other menus? What does this mean? If I pick another menu, am I changing the position of this menu relative to that menu? Am I inheriting some ordering properties from the menu I choose? This is totally unclear, so I leave it alone, and keep digging.

I click on the “Options” tab, and the third option I see is “Item ordering”, set to “Default”. I think to myself “Ah ok, this must be it”, change this to “most recent first”, and refresh the list (several times). No change.

Now the second most obvious fix has failed, and at this point I refresh the Joomla cache just in case it was enabled for some weird reason. It was a long shot, and of course, didn’t resolve the problem.

Going back to the menu settings, I decide to dig a little deeper. The options tab was pretty extensive, so I scroll down. At the bottom I see another “Item ordering” dropdown, but this one is set to “ordering” as the default selection, not “Default” like the drop down at the top. Odd that there would be two configuration options with the same name, with different defaults, one at the top and one at the bottom. I think to myself “eh, it’s Joomla, weird is normal”. I change this setting to “most recent first”, and then refresh the list. Still no change.

Getting a bit frustrated, I return to that menu options tab and decide to inspect the form field names to see if there’s something more revealing about them. When I go to inspect the redundant “Item ordering” field at the bottom, I notice that the change I had made didn’t even save — it had reverted back to “ordering” as the selected option. So I change it again, and this time scroll back down after saving to see if the change stuck. It didn’t. This field was bogus field that didn’t actually do anything. More time wasted…

Without recapping my entire debugging process, the end result was that I finally solved the problem 2 hours later. To this day I don’t even know what I did, it just started working. So what was a 30 second estimate, turned out to be a 2 hour task.

My estimate was 24,000% off