Managing Enterprise Integration Portal Projects

Performing large enterprise integration projects, spanning many different teams, and technologies, is a formidable challenge. This remains true, whether you use Scrum, or waterfall — there’s no magic process. Having said this, I’d like to share some things I wished I was more aware of, before embarking on a recent large integration project, for a SharePoint extranet portal.

Map Team Dependencies Using A Tool

Because enterprise integration projects span many teams, you will quickly realize that your standard Scrum board, that always served your solo development team so well, isn’t sufficient anymore. Combine this with some teams residing at different locations, and you have a situation where not only Scrum’s task board isn’t working that well anymore, but also many of the other Scrum techniques that depend on face-to-face interaction, like stand ups and Scrum of Scrums. What you need is a tool to connect each team’s work to the other teams they depend on, to create an inter team and story and task dependency graph. This way it becomes very easy to understand each team’s contribution to the overall completion and delivery of a work item.

At the start of each sprint the teams without any dependencies on other teams enter their stories and tasks first. When some of their stories have been entered, other teams that depend on them can start entering their stories in turn, and so it continues until all the teams have mapped out their work. For example Team Foundation Server (TFS) allows you to setup parent-child relationships between tasks and backlog items. By the way, I see Visual Studio-TFS Online now provides a free basic plan to 5 users. So, no more excuses for not using a proper Application Lifecycle Management Tool (ALM) ;-)

Code Reviews As Acceptance Criteria

A good way to increase the quality and knowledge of the application, is to make peer code reviews part of each story’s acceptance criteria. Doing lots of small code reviews, creates a faster personal feedback loop, compared to large code reviews every couple of days or weeks, involving the whole team. Because the code reviews are being done more frequently, mistakes in the code are identified more often and fixed sooner.

Try to pair developers with different experience, backgrounds and skills (i.e. try to avoid the same people reviewing each others code all the time).

You Need More Environments Than You Think

A system’s environment refers to all the supporting services and hardware, servers, and network infrastructure that an application depends on. For large, complicated integration projects you definitely want to make sure you have enough environments to make sure the application is thoroughly tested beyond the developer’s local workstation. For these projects the environment introduces a significant number of variables that impact on the application’s performance, security and general operation. What makes things more complicated, is that many of these infrastructure elements are not available on the developer’s local workstation.

The following system environments are recommended, to ensure any issues created by a new environment configuration, can be identified and quickly fixed early in the process:

1. A local development environment where the application gets developed. A good way to standardize this environment is to create a Virtual Machine (VM) that can be copied and used by all the developers. This environment is owned by each individual developer.
2. A development environment where the application can be tested by the development team. If you have more than one team working on the same code base, then you will actually need a development environment per team-code-branch, so that each team can test their version of the application independently of the other one. This is also the first environment where all the separate teams will integrate their applications and services. This environment is collectively owned by the development teams.
3. A Quality Assurance (QA) or acceptance environment where the application can be tested by a dedicated test team. This environment is owned by the dedicated test team.
4. Obviously production, where application’s customers will use it. This environment is indirectly owned by the application’s customers and the system administrators.
5. Highly recommended is another environment that mirrors production, so that problems in production can be diagnosed and tested in the same infrastructure configuration as production, without affecting the live environment in the process.

Make sure that all these environments, except the local workstation one, is part of the application life cycle at the client’s premises and inside their infrastructure.

Plan Sprints, Even If It’s Waterfall

Often the overall methodology used will be prescribed, and even if you can change this, it will take a lot of time to change an entire organization’s modus operandi. Usually corporate business prefers waterfall, and developers agile. It’s a lot easier to take control of development type tasks and apply agile practices to them. One such task is estimates. Waterfall estimation usually specifies a single ‘expert’, often outside the development team, to estimate work (this has been proven not to improve the accuracy of estimates though). Agile requires the entire team, that will do the actual development, to be involved. Therefore even though business might require a solo expert to do the estimates, the truth is they usually don’t care how you come up with the estimates, as long as they are accurate, and provided in a timely manner.

We found it very valuable doing group estimations using planning poker, even if the final estimate is given in hours or days, instead of story points. There’s an enormous amount of value in the discussion that goes into coming up with an estimate:

• Results in more accurate estimates because:

1. The actual people that will do the work gives them. This has a tremendous impact on how long something will take to get done. A bigger range of numbers is used to produce the final estimate. Statistically making for a sounder figure.
2. Facilitates critical thinking about the existing and design and technical implementation of the product.

• Improves shared ownership of work, and cooperation between team members.
• Improves learning about the product’s existing and future stories.
• Lowers personal barriers to communicate and debate.

It’s a story for another day, and a work in progress, about how we convert between story points and hours.

Estimate, Allocate and Manage Testing

Again, one of the things waterfall specifies is that testing is done as a separate activity, and not as part of the development process itself. While it’s definitely a good thing to have a separate QA test team, testing the product before it gets released, not doing this earlier during development is dangerous. It’s highly unlikely in an environment like this, that business will approve a dedicated tester for your team.

Well, it will take some effort to convince the developers, but add a testing activity as a percentage of the development effort to estimates. You now have some hours per sprint that you can use to allocate a developer to, to perform dedicated functional testing of the application during development.

If you don’t religiously guard against it, getting developers to perform functional testing, will lead to test corruption. Test corruption happens when a developer uses time allocated for testing, for other tasks, primarily developing new features and fixing bugs. The best way to manage this, is to allocate the testing activity on a set schedule to a developer, and ensuring she doesn’t do any development work during that time. Consider using a separate test computer without any development tools on it. This will make it physically impossible for the test-developer to do development work, and indicate to the rest of the team that she is performing functional testing, and should not be involved with development work.

Clarify The Meaning Of “Percentage Complete”

Communicating “how far” you are with a story or task is a standard requirement of any project. But what does it mean when you say you are “50% done”? Take some time to think what your typical development workflow consists of and what essential, but often forgotten tasks, you’d like included in the development of each story. This may consist of things like:

1. A story’s sub-tasks
2. Acceptance criteria
3. Developer and tester functional testing
4. Deployment

For example, you would probably want to do a peer code review before a story is marked as complete, but after all or certain types of tasks have been done. So maybe a code review being performed is 80% and code review corrections are 85%. Or perhaps, before the coding of each story commences, you require the developer to have a quick design session with the architect and/or one or two of his colleagues, so this will be 10% complete. Maybe you want a developer to deploy the new feature to a separate development environment, and perform his own functional testing, before the story is handed over to the team’s dedicated tester. What percentage complete would these be?

This will help each team member understand what is expected of him to complete a story, and the project manager and other team members know what is meant when he says he is “90% done”.