Software Archaeology: Probing the Past as Foundation for the Future
As a consultant and author, I often receive emails posing questions about the challenges people face dealing with requirements issues on their software projects. Here’s a question on an issue many teams encounter: replacing an existing software application.
George’s Question
“I am responsible for implementing a new system to replace a mainframe application that has been in production for a very long time. We have no documented requirements for the current system, the original developers are long gone, and our user community has since been changed out.
“Does it make sense for us to document the requirements of the current system as a prerequisite to documenting the go-forward requirements? Or should we disregard what we now have in production and document the go-forward requirements from scratch?
“Reverse engineering the current system might be a good way of kick-starting the requirements process and getting our users to think about requirements in general. But I’m concerned that such an effort will result in capturing requirements that are no longer pertinent, and therefore a possible waste of time and project dollars.”
My Response
In most cases fully reverse engineering a complete set of requirements from an existing system is not a good investment. However, you don’t want to lose the knowledge that is embedded in the current application. Selective reverse engineering is the best approach. I call that process software archaeology.
Some parts of the system that you’ll re-implement in the new one are worth documenting in considerable detail. Other portions need to be reverse-engineered just to a higher level of abstraction — to use cases, for instance, rather than to detailed functional requirements. And you can ignore portions of the existing system that represent obsolete functionality you’re certain you won’t be replicating.
I suggest the following strategy for this sort of situation:
- Go through the major features or functional areas of the current application. Identify those you’ll need to implement more or less intact, those you’ll replace with similar but modified or extended functionality, and those you won’t include in the new system.
- Document use cases from the current application that you will re-implement in the new app. Thinking at the abstraction level of use cases captures the user-needs perspective, while leaving open the specifics of all the functionality you’ll implement in the new system. You might not need detailed specifications for those use case if developers can gain a full understanding of them from the existing system. Reverse engineering use cases also facilitates developing a comprehensive set of test cases for the new application. One of my clients did this on a huge system that was the core of their business; it worked very well for them.
- Identify new use cases that the current application does not address but the new one must. This is essentially a gap analysis. Plan to ultimately specify requirements for the new capabilities to the same level of detail that you would for a brand-new application.
- Identify alternative and exception flows for each use case — old and new — to help you establish priorities for implementing the new system’s capabilities. Some use cases are more time-critical than others, and alternative flows often have lower priority than the normal flows. This analysis helps you invest your resources to get the new application online as quickly as possible and then enhance it over time in a rational way. Frequency of usage is one factor to consider when establishing priorities.
- Most applications contain considerable functionality that is not obvious to end users. These unobvious requirements include things like input data validations, internal auditing or transaction recording, access logging for security, and error handling. Specify such functionality from the current system so each developer doesn’t have to glean the knowledge from old code or try to infer it from the system’s externally visible behavior.
- Beyond functionality, look for other knowledge you can harvest from the current system that might not be documented elsewhere, such as business rules. Some of those rules will apply unchanged to the replacement. Others might need to be updated to reflect current business policies. Some new business rules probably will apply to new functionality you’ll be adding. This is a good way to begin growing a repository of that critical enterprise business knowledge that often exists distributed in various peoples’ heads.
- Specify important quality attributes that will pertain to the new system. These could be the same as those in the current application, or they may represent enhanced performance, usability, security, or other qualities. Avoid specifying performance in such vague ways as saying “The XYZ function must execute at least as fast as in the previous application.” That’s not verifiable unless you include enough information so the developer knows exactly what the expected performance requirements are.
- If you don’t already have one, build a logical data model based on the current application. You might end up fully re-implementing the current physical data model, but there probably will be some changes. A detailed data dictionary again represents critical corporate knowledge that likely is not well documented anywhere else, so record that knowledge for posterity.
- Identify any data migrations you’ll need to do to move data from the old system into the new. There might be a whole subproject, with its own requirements, for such transition requirements if the database or file structures change significantly. This is also a chance to clean up the old data: resolve duplicates, purge obsolete or incomplete records, add those fields and data-manipulation functions your users have always wished they had available.
Avoid the all-or-nothing syndrome in which people feel like they either need a complete and highly detailed reverse-engineered requirements set or they don’t need to pull anything out of the current system. There’s valuable knowledge there worth harvesting. Use your judgment to determine what aspects of the current system are most important to specify in detail, which can be left at a higher level of abstraction, and which you can omit entirely.
=====================
Karl Wiegers is Principal Consultant at Process Impact. If you’re interested in software requirements, business analysis, project management, software quality, or consulting, Process Impact provides numerous useful publications, downloads, and other resources. Karl’s latest book is The Thoughtless Design of Everyday Things.
