Building Artificial Intelligence Solutions: Agile Waterfall?
written by Derrick Willard
One question customers ask us at the KC Augmented Intelligence Lab (KCAIL) is how the process of building model-based solutions is different than the various methods used to create general IT applications. This article explores some of the similarities and differences we see between those two.
Many who have been around IT for any time are familiar with traditional methods such as Waterfall with its rather rigid progression of logical steps when completing the software development life cycle. This was the approach popularized in the 1970s and is particularly useful if “requirements and scope are fixed, the product is itself firm and stable, and the technology is clearly understood”.
These same customers may also be aware of newer approaches such as Agile that attempt to address the inflexibility of Waterfall by providing validation of the customer’s intent and the correctness of the design much earlier in the process when failures in understanding are much less costly to correct. Rather than Waterfall’s practice of publishing code for user acceptance testing at the end of what can often be a many month effort, Agile development thinks in terms of 2 to 4 week scrums in which a full cycle of design, code, test, and deploy occurs. Agile “advocates adaptive planning, evolutionary development, early delivery, and continual improvement, and it encourages rapid and flexible response to change”.
Which approach is most often used for data modeling projects? Many data science teams, including KCAIL, use variations of the Cross-Industry Standard Process for Data Mining (CRISP-DM) model. CRISP-DM dates back to 1996 and is currently championed by IBM along with its modified version named ASUM-DM (Analytics Solutions Unified Method for Data Mining/Predictive Analytics) that adds a deployment set of phases to the development oriented CRISP-DM model.
Here’s a popular illustration from Kenneth Jensen of the CRISP-DM phases with an explanation of each:
Business Understanding includes:
1. Determining the business objectives — what problem does the customer wish to solve?
2. Assessing the situation — including requirements, assumptions, risks and cost/benefit
3. Determining the methodology and project plan
Data Understanding includes:
1. Collecting, describing, and exploring the initial data
2. Verifying data quality
Data Preparation includes:
1. Selecting, cleaning, constructing (as needed), and integrating data
2. Formatting data
Modeling includes:
1. Selecting model techniques
2. Generating a test design
3. Building and assessing models
Evaluation includes:
1. Evaluate results
2. Review process
Deployment includes:
1. Planning the deployment
2. Planning monitoring and maintenance
3. Final project review
So, is CRISP-DM more Waterfall-ish or Agile-like? As a development oriented method, it can be adapted to either approach. Where there is ambiguity (you don’t know what you don’t know), a more Agile approach will be less risky and preferable. Where the scope and requirements are firm and the team is well experienced in the subject domain, a more traditional approach may be permissible. In either event, note that the diagram for CRISP-DM above has certain feedback loops (i.e. between business and data understanding, between data prep and modeling, and between evaluation and business understanding) that bring an element of Agile’s iterative approach.
One area where we find CRISP-DM lacking is specific support for wrapping a data model with other development such as a mobile app or web site, API integration, and database integration. These activities however fall into fairly typical IT development models.
Lastly, regardless of the methodology employed, a proper understanding of the business problem at hand is essential between the customer and developer if a useful automated solution is to be crafted. Frequent re-validation is also necessary as many problems require progressive elaboration and, with the passage of time, things do change.
