Some Advice for Developers

What a veteran developer has learned about building software

Please allow me to introduce myself. My friends call me Dick and I’ve been building software since 1972. I’ve filled many roles including coder, software engineer, team lead, designer/architect, Senior Architect for Database Replication, Director of Research & Development, VP of Software Engineering, consultancy manager, and even CEO of a successful software company.

Things have changed a lot since I started — and there is always something new to learn — but I’ve enjoyed the whole ride and have no plans to stop.

Before I was a software developer, I was an officer in the US Army. I’ve led troops in combat and that experience taught me some important lessons that have guided me as a developer:

1. Life is a team sport and most important things get done by groups of people.
2. A shared sense of purpose and effective bi-directional communication — both vertical and horizontal — are essential for groups to function well.
3. Planning is important and plans need to be continually reevaluated and updated as feedback dictates — but plans are only the best guess for getting there, not the objective itself.
4. Self-delusion and wishful thinking can ruin any plan.
5. Failure is not an acceptable option.

At university, I trained as a biologist. That training has also influenced how I think about software system complexity and how to manage it.

The most complex systems on this planet are living organisms — and living things are self-organizing. Think of beehives, ant colonies, termite mounds, and even our own brains. The words “organize” and “organism” have the same Classical Greek root.

Ten Lessons Learned from Building Software

1. Software development is accomplished by teams: So it is important that the architecture, design, and coding practices of a system clearly and unambiguously communicate their organization, purpose, and intended function to the people who have to work with them.
2. Complexity is the enemy of good software: So good architecture, design, and coding should seek to manage and minimize complexity. If you haven’t made the individual parts simple, you really haven’t finished the job.
3. Listen to the experts: We developers should be experts on how to build software, but we are not the application experts. We need to listen to the real experts on what the application is. Developers and application domain experts — not intermediaries — must collaborate to figure out what a software application should do.
4. Architecture is like plumbing: It defines component models and how components are connected. Architecture shapes the design and coding of the components that implement application functionality and the interfaces that connect them. Architecture informs design and design informs coding.
5. Components are building blocks: They describe the things, or aggregates of things, that make up an application. They are the message processing filters of an underlying pipes and filters pattern. Small, clearly bounded components with explicit interfaces are easier for people to understand, code, and test. I recommend the composable services pattern because it is easier for people to conceptualize building applications out of things than out of functions.
6. Connectors are like mortar: They glue the components of an application together. They are the pipes of an underlying pipes and filters pattern that can support high levels of modularity and flexibility, and make the components of an application system easier to understand, extend, maintain, and reuse. I recommend message orchestrators for connecting components.
7. Messages are the payload: They flow through the connectors to be processed by components. They tell components what we want them to do and provide the information they need in order to do it. Messages should be deliverable synchronously, asynchronously, and as events. I recommend using RESTful messages and components.
8. Application organization is critical: For the solution, we can look to self-organizing, self-configuring systems, the way nature copes with complexity. In this way control is distributed over whole systems and all parts contribute to the resulting functionality, as opposed to centralized structures that are often dependent upon a single coordinating entity.
9. Change is inevitable: One of the most effective ways to manage change is to use component models that a) make individual software components easier to understand and change, b) insulate software components from changes to other components, c) minimize the potential interference among teams working on different parts of the system and, d) simplify the delivery of new and updated software components.
10. Failure is not an option: Software development project failure is not an option for many reasons, including company and customer financial implications, company and development team reputational damage, and the strategic importance of many software projects to companies and customers. To avoid failure, performance against plan must be continually and accurately evaluated and the plan adjusted to fit reality and deliver the best results achievable within acceptable limits. If acceptable results cannot be achieved, then the project must be re-planned and a workable balance of benefits, resources, and time agreed. Rarely does adding people (except for missing skills) to a project positively impact the schedule. Within limits, fixing identified problems (if possible) and delivering fewer benefits (if acceptable) have the best chance for success.

The true measure of a good plan is not whether its execution transpires exactly as intended, but whether it facilitates effective action in the face of unforeseen events — and encourages exploration and problem solving as needed.

What Makes Application Development Different?

To succeed, it is critically important that all the people managing, funding, or participating in an application development project understand that application development is an exploration that requires an iterative process of discovery, learning, and problem solving. Treating it as a cut-and-dried series of tasks that can be precisely defined in advance, estimated, planned, and then executed will doom it to failure.

Experience tells us that application development has many faces:

1. Navigating the Unknown — Just like explorers chart unmapped territories, developers embark on projects without knowing every detail or challenge ahead. They must navigate through learning application requirements, frameworks, languages, and technologies, adapting their knowledge and skills to overcome unforeseen obstacles. Every project is different. Despite what some would like to believe, every new project is a journey into the unknown. Treat it with respect.
2. Innovation and Discovery — Exploration is driven by the desire to discover new lands and possibilities. Similarly, application development often aims to innovate, whether by solving a problem in a new and better way, creating a unique user experience, or integrating the latest technologies to improve performance and functionality.
3. Iterative Learning — Explorers don’t always succeed on their first attempt. They learn from each part of the journey, refining their strategies and approaches. In application development, this iterative process is mirrored in the design, develop, test, and deploy loop, where feedback and real-world usage inform continuous improvements and refinements.
4. A Team Expedition — Exploration is rarely a solitary endeavor. It involves teams with diverse skills and knowledge. Application development also relies on multidisciplinary teams, including designers, developers, project leaders, and application domain experts, each contributing their expertise towards the common goal.
5. Mapping the Journey — Just as explorers map their journeys for those who follow, documentation in application development serves to guide current and future developers. It helps maintain an understanding of the application’s architecture, dependencies, and decision-making processes, ensuring the application’s longevity and scalability. That documentation should reside within the application’s own artifacts (discussion and design documents, diagrams, source code, etc.) where it will be seen and used by the people who need it most.
6. Dealing with Uncertainty — Explorers face uncertainties and must make decisions with incomplete information. Application developers encounter similar situations, whether dealing with changing requirements, emerging technologies, or unforeseen user needs. Success often depends on the ability to make informed decisions quickly and to pivot as needed.
7. Resource Management — Explorations require careful planning and resource management to ensure the journey can even be completed. In application development, managing resources such as time, budget, and developer skills is crucial to delivering a project successfully within its constraints. Planning is a means of navigating — but not the primary purpose of — an application development project.
8. Common Sense — Explorers are optimists or they wouldn’t seek the challenges of exploring the unknown. However, the optimistic nature of software developers can be a double-edged sword. On the plus side, it can drive innovation and relentless problem-solving — but it can also lead us into the trap of underestimation and overpromising. If we break our plans into tasks of no more that 2 person/weeks (80 hours) each — and always measure them as either 0% or 100% complete — we will find it harder to be overly optimistic and fool ourselves about how far along our journey we actually are.

Application development projects are not like manufacturing or construction projects, where it makes sense to eliminate uncertainty as early in the project as possible.

With application development projects, it is impossible to eliminate uncertainty.

Like explorers, we may have a good idea of where we want to go — but we cannot know exactly what where looks like — or exactly how to get there. We must navigate through the obstacles in our path — as we learn the route — while not running out of resources on the way.

To avoid failure, performance against plan must be continually and accurately evaluated and the plan adjusted to fit reality and deliver the best achievable result within acceptable limits.

Technology, often perceived as the core requirement of software development, is only one facet of a much broader discipline. Software development is more profoundly impacted by how effectively the people involved can communicate and coordinate with each other. Better teamwork usually wins!

Companies that are successful at application development have learned not to fear uncertainty but to manage and thrive on it.

If you found this article useful, a clap would let us know that we’re on the right track.

Thanks!

Suggested Reading:

• Is OOP Relevant Today, does object-oriented programming still have a purpose?
• Composable Services, making the components of a system fit together.
• The Magic of Message Orchestration, the wiring that connects composable components.
• Designing a REST API, messaging using state representations beats remote procedure calls.
• Strategic DDD by example: subdomains identification, finding your components.
• Building Software Systems, breaking through the complexity of developing software.
• Great Software Design, the impact of design on good software.
• Uncertainty and the Art of Business Planning, what we can learn about software development from an Army field manual.