Evaluating Learning in 3 Questions: What? How? When?
This is the year. My New Year’s resolution. I am going to write a reflection on the class I teach during the spring semester. I figure that if I tell you this, then you’ll hold me to it. I’m in education — I teach — I do research — I actually think of teaching as research (TAR), but like real tar, TAR is sticky. Part of being a researcher, and an education is being a reflective practitioner. Yet, finding the time to reflect is a challenge. It’s like creating a time capsule. It’s something that takes extra time in the now, but can be valuable in the future.
I teach an education class at MIT. Yes, MIT students can take education courses, and can get certified to teach through the Scheller Teacher Education Program (STEP)*. This is the third year I am teaching “Understanding and Evaluating Education” (CMS 587/ 11.125) at MIT. The course counts towards the writing requirement and is always overenrolled, so there were a lot of students in the room. The class opened with the standard lineup: I passed out a sign in sheet and everyone signed in, introduced myself, our writing advisor Janis Melvold, and my co-teacher Kristina Heavey, and I reviewed norms, expectations, what they will be doing, and how they will be assessed. There were some questions, but not many. We took a short break midway through and I told everyone to get sticky notes and a marker. The thinking always begins when the sticky notes are brought in.
Because MIT doesn’t have a full teacher education program, this course attracts students who are just beginning to explore the profession. Few have teaching experience, so we build from the ground up. I asked the students to answer three questions:
What is important to learn?
How do you learn it?
When do we know that we’ve learned it?
This is a set of questions that I borrowed from Justin Reich, who has taught the course in the past. I wrote the three questions on different parts of the board, everyone wrote their responses to all three questions on sticky notes, and then we filled the board where I had written the questions. I split the groups into three and each third picked one question, reviewed the sticky notes (5 min), and then begin to group the notes into categories. Students were bunched up at the board, so I asked them to rotate (“like convection currents”) so everyone could have a chance to see and to organize.
After it appeared the organization was complete, everyone went back to their groups. Realizing that I hadn’t warned them that they were going to report out, I asked “is it ok if I select the person to report out on the sticky notes” and they said “yes.” So I had group 1 raise their hands, and I selected one person to be the spokesperson. We did the same for groups 2 and 3. I gave them a couple of minutes to discuss with their reporter before coming to the front of the room to share what the themes were.
The results:
What is important to learn?
First, the students indicated that the most important stuff to learn is not content, but skills such as communication (mentioned on 11 sticky notes) and teamwork, and processes such as“learning how to learn” (11). When I asked the class their thoughts about who decides what is important to learn, students mentioned parents, teachers, schools, communities, nations, and the individual. There was also a little section of sticky notes labeled “STEM” — the MIT students evinced their priority for STEM by including logic, computer science, and math as important. We joked about it, but also noted that our priorities for what is important to learn reveal our relative priorities for the domains themselves.
How do we learn it?
In thinking about “how do we learn it”, students mentioned both practice (11) and social interaction (11). Practice takes the form of watching others, doing examples, repetition, and “active practice” — doing something with intention. Social interactions that help the learning process included interacting with peers during group work. They also identified interacting with beginners through teaching as a source of learning, as was hearing different viewpoints. Some students also noted the importance of intentional practice, reflection, and feedback. We will explore the concept of “practice” throughout the semester.
When do we know we have learned it?
Being able to apply what you learned to a new situation or problem was the most commonly mentioned evidence of learning (9). This ability to transfer understandings in one domain to a different domain is considered to be a gold standard in demonstrating learning. However, transfer is difficult to achieve for two reasons. First, novices have a difficult time recognizing the underlying core themes when they are surrounded by potentially confusing surface level features. A favorite example is when physics professor Eric Mazur at Harvard spent a semester talking about forces using examples from baseball. On the exam, he asked about the exact same concepts but used football as an example, and the students claimed “you never taught us this!”. The second reason transfer is challenging is that finding an appropriate approximation of the skill can be difficult. We want to know that a medical student studying to be a surgeon can do a heart bypass, but we don’t want to test it on a live patient! At the Teaching Systems Lab (TSL) at MIT, we design playful and engaging approximations of practice for learning key skills in teaching. We call these games and simulations practice spaces, and I plan to use a few of them in class.
Students considered confidence and comfort with the material (8) as another indicator of learning, as well as being able to pass on your understanding to others (5). A few students mentioned self-reflection (4) as evidence of learning, and concrete application (4), such as passing a test, getting into MIT, or “cooking a meal without burning the house down”. They also noted that learning is an ongoing process.
In my wrap up, I explained how “what is important to learn” in school is often delineated by the states, as states have more power in education than the federal government. I mentioned that we would cover how people learn through investigating learning theories and ideas from educators, psychologists, cognitive scientists, and policy makers.
The third question — how do we know we have learned it — is often the most challenging one to answer. Evidence of learning is tough! I can see if you’re facing the right direction or nodding (and not nodding off), but it’s challenging to really know what your students know. Both challenging and essential!
These three questions form the core mission of the course — to explore what counts as evidence of learning requires an understanding of what is being learned and how. I always enjoy teaching this class — I learn a lot from the students, and appreciate their honest ideas and different viewpoints.
There was one sticky note in the far left corner. I almost didn’t notice it, but someone raised their hand and asked “what about that one?”
Ah, the insightful outlier, reminding us that context is always important……..
