Grading from the Bottom-Up: Looking at What Emerging Designers Learn, Not What They Make
Aaron Ganci, Associate Professor of Visual Communication Design, Herron School of Art and Design, Indiana University, IUPUI
Letter grades are problematic in design education. Design activity often does not fit neatly into the traditional A–F grading scale, where there are distinctly right and wrong answers. Victor Papanek told us this almost 40 years ago in “Design for the Real World;” he said: “Design, as a problem-solving activity can never, by definition, yield one right answer: it will always produce an infinite number of answers, some ‘righter’ and some ‘wronger.’” With this in mind, it is essential for design educators to reconsider what role assessment has in contemporary education.
If there are no right or wrong answers, how can we assign a grade to a student’s work? Of course, there are some basics that we can easily assign quality: the traditional elements of visual design (hierarchy, balance, color contrast) and implementation of platform or OS standards. However, there are many other factors at play that dictate the ‘rightness or wrongness’ of a solution. These are things that often get overlooked when considering a “final” solution and include: a student’s baseline ability, the complexity of design constraints, the fuzziness of problem contexts, multidisciplinary and collaborative group dynamics, instructor learning objectives for the activity, and, I would argue, the unique factors of a student’s life outside school. These factors all contribute to a student’s performance and, perhaps more importantly, their learning outcomes in a project.
Our Relationship With Grades
Grades have acquired outsized importance to which we tie a lot of complex meaning. They are the fossil fuel that keeps everything moving. Everyone involved in the system is evaluated on student grades (students, instructors, institutions, legislatures). Naturally, with so much meaning and emotion tied into grades, they cause a lot of anxiety. My students’ struggle with this anxiety is palpable, and I see a direct connection between it and unrealized potential in the studio. How can we take some of the pressure off students and enable them to grow as much as possible as designers?
An Issue of Equity
There is one other issue at play here; students enter my studio at different levels. Baseline abilities vary greatly. This is partly due to the type of institution I exist within. As one of the only professional art and design schools in our state, Herron School of Art and Design attracts students from every demographic and region in the state. Indiana has seen budget stagnation for public education; the arts are often one of the first things to get cut. Local school districts are responsible for funding things like the arts only if they are able. Of course, the result of this is that students from wealthy districts have stronger foundations in art and design principles and software. In contrast, those from poorer districts lack those basic skills.
Another factor related to equity is that students have very different demands placed upon them outside of school. Again, these differences are mostly a result of socioeconomic status. Students that do not have outside financial support have to work more, sometimes 30+ hours each week. We have to take these factors into account to not further disadvantage these students. With these factors in mind, I would argue that looking at final outcomes is an unproductive way to measure a student’s ability or learning.
A New Way to Grade
It has been clear to me for some time that traditional grading methods are not effective or equitable for design studios. Over the last three semesters, I have been developing a prototype of a new system that I think works better. The foundation of this system is built on the idea of freeing ourselves from letter grades and place emphasis on qualitative, reflective assessment. Of course, I thought of this as a design challenge and set the following criteria:
- De-emphasize (or remove entirely) a letter grade.
- Emphasize divergence within the design process.
- Help students focus on their growth rather than compare their final output to others.
- Empower students to assess themselves.
My Grade System Prototype
Reflective assessment has been done in design for quite some time and with much success (See references below). I based my new system on a process similar to Ellmer’s Reflective Framework.
Through this framework, designers are asked to advance their learning by abstracting and reflecting on their process and output. It separates the activity of design from the designer’s understanding of their process and ability. With this in mind, I am proposing that this become the basis of how we evaluate students. We need to fully abandon quantitate measures in favor of qualitative reflection. This is what I am seeking to create in my studios.
The language I used in my syllabus will provide the clearest overview of my prototype. This is the exact text that students received at the beginning of the semester:
Grades are a little different in this course. Traditional letter grades hinder creativity and place emphasis on task completion rather than creative investigation. With this in mind, your assessment in this course will be focused on your learning rather than your output. At two points in the semester (once around mid-term and once at the end of the semester), you will be asked to complete a self-reflection that will facilitate a one-on-one discussion with me. At that meeting, we will discuss what grade we both believe is appropriate. With all this in mind, expectations for deliverables will always be clearly communicated, and your participation in those activities will be considered when determining your grade. Feedback from me, our TA, and your peers will be provided in both structured (rubrics) and unstructured (critique) venues throughout the semester.
In this statement, I have entirely de-emphasized letter grades and instead, empowered the students to understand and assess their learning. This new method does require educators to do a little extra work up front. I will outline those steps below if you would like to implement this in your studio. You can view my full syllabus for this course here: https://www.dropbox.com/s/ze9y6usegxnzpv0/Ganci%20-%20Herron%20V408%20UI%20-%20Syllabus.pdf?dl=0
Step 1: Be transparent and honest. Tell your students that you’re trying a new way of grading and that you are doing it to improve their ability to learn. Students will not be comfortable working in this system. I tell my students that I DO NOT care what their final outcome looks like. In fact, if they are doing it right, it won’t look as polished as it could. That tells me that they abandoned ideas along the way and had to start over. Use language that will shock them out of their typical association with classwork and grades.
Step 2: Clearly define your learning outcomes. This new system hinges entirely on the learning outcomes. Provide students will clear outcomes that describe what they will be able to do at the end of the term. With those in hand, you can ask them to evaluate how well they can do those things at the end of the term. This system allows you to mix messy, ambiguous skills with industry-standard hard skills. Three of the objectives of my UI Design course illustrate this:
- Use a concept scenario as a tool to describe an interaction design concept
- Prototype solutions using industry-standard tools (Sketch/Invision or XD)
- Discuss the ethics and effects of personal data collection
Step 3: Increase the number of feedback venues. Freeing everyone from grades frees you to be honest about what’s working or not. When a student is trying to polish a lousy idea in hopes of a better grade, they are less receptive to critical feedback. Students might need to start over, which is now okay. These settings also provide a space where you can really critique hard skill details. Think of these are a series of mini check-ins rather than exhaustive end-of-project critiques. I try to have each student talk about their work in front of a group once each week.
Step 4: Write good reflection prompts. If you wrote good learning outcomes, most of the work is already done. I ask students to reflect on each learning outcome and evaluate their performance in each one of these four categories:
- In-class participation (attendance + tardiness, taking part in discussion/crit)
- riskiness/continuous improvement (willingness to diverge dramatically, willingness to abandon ideas if they weren’t working)
- technical skills (create visual designs at a high level, utilize the principles of design effectively, craft, etc.)
- overall assessment (“my performance in this class can best be summed up as”).
This last question is where they give themselves a grade. Give students at least one week to complete this reflection and ask them to be very thorough. Tell them to spend at least an hour with the reflection. I provide my students with sub-questions that get at the more practical elements of that outcome. Frame it in ways that help them reflect on their learning. Here is an example from that same UI course (view full reflection document here). [https://www.dropbox.com/s/bx8i8mjbdkcbuv9/Ganci%20-%20Herron%20V408%20UI%20-%20Final%20Reflection.pdf?dl=0]):
Learning Outcome 3: Prototype solutions using industry-standard tools
What tools did you use to design your app? What have you learned about XD or Sketch/Invision as prototyping tools? What can you do now that you weren’t able to do at the beginning of the course?
Step 5: Block off the last week of the term for reflection. This was the most significant change for me. You lose an entire week of work, but it is essential to have plenty of time to have in-depth conversations with each student. This is where you can talk about their self-assessment and agree on a grade. Before you meet, review their written reflection and note where you observed differences. Of course, you can still look through their design output. This format just gives you more space to go deep into the rationale, process, and struggle. These sessions give you a very nuanced idea of what each student learned and how their ability grew.
Student Feedback and Improvements
I conduct an informal, anonymous survey with my students after the term to gauge their thoughts about the new system. There are several things that students like about this new system:
“I enjoyed the flexibility and that it allowed us to be more daring in our thoughts and designs. I loved the reflections and 1 on 1 meetings. It was such a good way to get direct feedback, but also to be able to defend your thinking and have a discussion.”
“It encouraged better and more personal dialogue with the professor. The one on one reflections at the end gave a good platform to ask questions regarding feedback or general career guidance.”
This second quote is an unintended consequence but is welcome. By focusing on their design knowledge in the course, students seem to be better able to reflect on their professional aspirations.
Most of the students I surveyed enjoy the new system. However, there is always room for improvement. Here is a snapshot of critical feedback from students.
“I think that the lack of normal grading may have allowed me to relax [my work ethic] a little bit. Students definitely need to understand and be ready for the social contract required to make this work.”
“I didn’t really know how well I was doing through out [sic] the semester and didn’t know where or what I needed to improve on.”
This tells me that I clearly need to integrate more structured review sessions in the semester. Students enjoyed the extensive one-on-one meeting at the end of the semester and want more of that individualized, honest conversation. Of course, it is always a challenge to provide this for large classes. The other significant change I will make in the next round is to ask the students to give me a grade range they think they fall within. Students were generally good about being critical of themselves (not everyone gave themselves an A; in fact, most did not). A grade range will provide a better conversation tool in the meeting. It is more effective to talk about the nuance between an A- and B than to just explain why they didn’t earn an A.
I have been pleased with this new grading system and encourage you to try it in your studios. Traditional grade systems do not work well in design for a variety of reasons. Removing the stigma of letter grades and focusing on a student’s learning establishes a more productive and equitable studio environment for our students (and ourselves).
References
Ellmers, Grant. “Reflection and graphic design pedagogy: Developing a reflective framework to enhance learning in a graphic design tertiary environment.” 2006 ACUADS Conference: Thinking the Future: Art, Design andCreativity, Faculty of Art & Design, Monash University & School of Art, Victorian College of the Arts, Melbourne, 27–29 September 2006, 1–10 (2006).
Lousberg, Louis, et al. “Reflection in design education.” International Journal of Technology and Design Education (2019): 1–13.
Morgan, Thea. “Philosophy of Design: Enabling reflection within PBL contexts in engineering design education.” DS 93: Proceedings of the 20th International Conference on Engineering and Product Design Education (E&PDE 2018), Dyson School of Engineering, Imperial College, London. 6th-7th September 2018. 2018.
Papanek, Victor J, and R. Buckminster Fuller. Design for the Real World : Human Ecology and Social Change. New York: Bantam Books, 1973.
Samuels, M N and Betts, J (2007) Crossing the threshold from description to deconstruction and reconstruction: using self-assessment to deepen reflection. Reflective Practice, 8 (2). pp. 269–283.
Tracey, Monica W., and Alisa Hutchinson. “Reflection and professional identity development in design education.” International Journal of Technology and Design Education 28.1 (2018): 263–285.
About the Author
Aaron Ganci
Aaron Ganci, MFA is a user experience and interface designer and an Associate Professor of Visual Communication Design at Herron School of Art and Design at Indiana University, IUPUI. He is a frequent consultant on the design of websites and software interfaces in the healthcare, public health, financial, and energy sectors. More than 100 million people have used a product that he has designed. In addition to professional creative activity, Professor Ganci teaches a range of courses in user experience design, user interface design, service design, AR prototyping, and a variety of graphic design topics. He also researches contemporary design practice, technology use in American culture, and the use of technology to personalize design artifacts.
