User Study 2: Synthesis Model of Educators
Affinity Diagram & Diagnostic Map Creation
Moving from the user interview sessions we hosted with stakeholders, we reread our notes and record transcripts to mark out the important points mentioned in the interviews, and built up an affinity diagram to group ideas and find potential problems. The colored post-its represented different stakeholders, blue for Mr. M, and orange for Dr. F. In order to better support our synthesized ideas on the post-it, we cut the key quotes from the transcript and attached relevant one at the bottom of each post-it. After we organized them into subgroups, we used pink post-it to summarize the main point for the groups of notes.
From the affinity diagram we found out that Mr. M has his problem mostly in four aspects: documentation, assessment, feedback and showcasing, while Dr. F had her concern mostly on assessment and showcasing. We combined their concerns into a set of diagnostic maps with four focus.
For documentation, educators are concerned about the documentation of students’ working process, and how to recall the specific stage of each and every group. Currently, Dr.F mentioned that there is not much topics around students’ documentations and instructors do not talk a lot about how to teach documentation. We believe students need to form the habit of actively documenting since both of stakeholders mentioned that documentation is critical in terms of both portfolio piece and assessment. Documentation can actually show how student think and map out the decisions.
For assessment, it is hard to evaluate both individual work inside a group and between groups. However, Mr.M does not treat this as a drawback since he knows students well enough to fairly assess their work and contribution based on their skill sets and abilities. In terms of feedback students received from others, they are not usually given authentic and useful feedback from both community and experts due to the format of presentation and the informal requirements about presentation feedback from audience. The main reason for this, school do not want to force audience to give feedback. But the tradeoff is students cannot usually get enough and authentic ones.
For showcasing, WT currently has some difficulties with their presentation symposium because of limited space, and also there are some difficulties demonstrating students’ work online.
Co-Design & Diagnostic Map Revision
We shared our maps with instructor to make sure our understandings about problem space and their concerns are correct. We got lots of useful feedback and suggestions from Mr.M on these maps. To allow us further follow the guidance of these maps, we want to address the issues we found form the co-design session.
For documentation process, our stakeholder Mr.Marx expressed that he was hoping for a suitable tool that would show powerful effect very soon, and also it needed to be unique. He doesn’t want to explore technology tools one by one, that was why he asked for customization.
For assessment process, we found out that we had been pre-assuming challenges for the course. Since nature of being flexible and designed to maximize students’ interested topic, students will be focusing on totally different topics and they would have various starting points. It is hard to assess the whole class with a set of fixed rubric, thus subjective grading is inevitable in this class. But just because of this characteristic, that students can get more targeted feedback and instruction. And since Mr.Marx worked really closely to student groups, he would be qualified to evaluate their work and progress, and finally grade them fairly.
For feedback, we found out that Mr. Marx really wanted his students to have more natural feedback from the audience. And also the current symposium in WT is at the end of the semester, so students would not realize the importance of documentation during their working process since they have not thought about putting their work together while working. Mr.Marx said it would definitely increase students’ attention on documentation to have a mid-term presentation.
For the showcasing process, Mr.Marx really doesn’t want the audience to compare projects, he thinks each and every project work has its unique meaning. That is why they did not use guidance tools for the symposium. But they are also open to better organization dynamics for showcase and demonstration.
This brought us to the end of user study 2 and we then move to understand the needs of learners in our next post.
