Step 5 of the Design Process: Solicit Feedback (closed loop systems)

Is it dry enough yet? No… Is it dry enough yet? No… Is it dry enough yet? Yes. *End Cycle*

Some dryers have a moisture sensor built in to replace the timer cycle. The idea is to have the machine decide when the drying should stop and not to stop simply on a timer. A timer has a fatal flaw as the time chosen might be the wrong time to end a drying cycle, too long for crispy clothing and too short for clothing not yet dry. A timer is fully arbitrary and is dependent on the user’s best guess to be the correct amount of time for that unique load. Hence, a moisture sensor has an advantage over a timer and this sensor is regularly probling to determine success.

Schools often use tools no more powerful than a timer on a dryer to decide when and how an intervention or initiative has finished or has reached its goal. It isn’t that the wrong tool is being used to decide when something has been accomplished, but rather, there isn’t often any decision made about how to determine success prior to beginning an initiative, assessment or intervention at all. Let’s take a classroom reading test as an example. An elementary grade level team has followed the design cycle and put on their very best design thinking-caps and have diligently made a wonderful effort to create their best reading assessment for their favorite unit. From a well designed problem through a few prototypes and the group is set, they administer the assessment, look over the results at the conclusion of the unit and pack it all away for next year. This is an open loop because there is no sensor being employed to determine success and end the design & implementation cycle. This open loop could also be visualized as a straight line of development.

“begin design process → define problem → brainstorm → develop solutions → end”

The design cycle didn’t finish after prototyping and initial implementation. First a quick reminder about the design process:

A quick reminder of the Design Process & article links:

1. Define the Problem (Verification vs. Validation)
2. Collect Information (Nodes and Links)
3. Brainstorm and Analyze (Modeling or Ideation)
4. Develop Solutions (Prototyping)
5. Solicit Feedback (Closed Loop)
6. Improve Your Design (Iderations, Iterations, Iterations!)

Step five suggests that we solicit feedback. Perhaps your initial image in your mind was that of a meeting with colleagues around a table telling you what they think. That is one critical form of feedback for sure, but it is only the beginning. Many different types of designers will test a new design in the real world, in a lab or simulated to look for very specific and measurable successes and to detect failure. In this way, designing is happening as a closed loop where a specific result is sought and the project will not conclude until the result is achieved. Same for the washing machine, the dryer moisture sensor is a closed loop because the drying cycle will not conclude until a desired result is reached.

Let’s look at some possible generic sensors that might be worth considering in this assessment example that could work for most assessments. Sensors are best tied directly to Step 1 and the well-identified problem.

1. Did the assessment measure what we were hoping for?
2. Did the assessment measure mastery as we had hoped?
3. Were students able to “game” the assessment?
4. Are any issues of bias, cultural or otherwise apparent?
5. Were our higher needs students able to use the test effectively?
6. Did it take longer for students to complete than had been anticipated?
7. Does our measurement tool (rubrics / criteria) match student work?
8. Have we accurately identified students who need further support?
9. Have we accurately identified students who are ready to move on?
10. Is the assessment efficient to produce, administer, grade and reuse?

Had this team of teachers sat down with a few sensors in mind, perhaps they would have made a few changes initially to their design but more importantly, those teachers would have had a critical eye to evaluate the success of the assessment when completed. The team being able to identify success and failure may not only help them continue to work on that assessment but help them start from a better place with their next test in their next unit, having learned from this test. Finally, when time is given to ensure that a great conclusion has been reached and the best design picked, you can be sure that the work you put into the process will stand the test of time. You won’t be fixing the problem all over again next year but advancing your work at your school beyond this point. You will evolve forward and meaningfully.

Originally published at skhooldesign.com on February 10, 2019.