The Positive Pressure from Peer Evaluations
An important topic in the Managerial Economics course I teach to MBA students is the economics of price discrimination. The theories learned in class are applied in a team-based project that analyzes the use of price discrimination in the commercial airline industry. For the past several terms, I have included a peer-review component to the assessment of the project.
While my use of peer-evaluation is a work in progress, my subjective feeling is that the overall impact of using peer evaluations has been positive: the overall quality of projects has improved, and the number of A or A- papers has increased. Prior to using peer evaluations, this project served as a sort of ‘dispersion grenade’ for the grade distribution in the class, with some teams producing exceptional projects but other teams not so much. Students report some of the following outcomes from the peer evaluations process:
- Improved sentence structure and a dramatic reduction in grammatical errors
- Paragraphs and information reorganized to improve the logical flow
- Re-arrangement of data tables and graphs to clarify and support the presented work
- Introduction of formal hypotheses to test the ideas being presented
- Alignment of the individual sections to make the report sound more like one ‘continuous voice’
Perhaps it is the positive peer pressure that has improved overall quality; perhaps it is the specification of a detailed grading rubric, which is obviously necessary for the peers to know how they are to assign grades, but that I had not previously provided. Undoubtedly these are important. But this is also a concrete analog to the ‘wisdom of crowds’ which we teach in Finance and Economics. A well-functioning market takes small pieces of information from a large number of participants and efficiently organizes and aggregates this information into highly-informative prices. With the peer evaluation process, diverse and constructive tidbits of feedback from everyone in the class produce projects that are of higher quality, easier to grade, and more enjoyable to read.
Because teams naturally delegate responsibilities for the project, the peer evaluation of two other teams’ projects forces each student to become sufficiently familiar with all aspects of the project by reading, commenting on and grading all elements of the project. At the same time, students who possess or have developed particular expertise may be able to provide constructive, specific feedback that not all students possess.
Several concrete aspects that are important to me
- Grading is done on an absolute scale, not a relative one. In my opinion, it is essential that this grading philosophy is communicated and executed to eliminate the concern that by ‘helping my classmates improve their paper, I am raising the bar by which my own work is evaluated.’ The absolute grading scale also forced me to define all of the essential elements of a “great paper”, and once in place has helped me to more efficiently and objectively grade papers.
- Each team is required to include an Authors’ Note at the end of their first and final drafts stating what they feel are the current strengths and weaknesses of the paper. In addition, they are encouraged to use the note in the first draft to identify areas where they specifically want feedback. Reading the Note not only helps me understand how the project evolved, but most importantly allows me to tailor my feedback on the final project to those areas where the team has explicitly recognized or asked for further help. It is one simple way of ‘closing the loop’ and giving students something to think about after the project, rather than simply leaving them with a final grade and a sense that this was yet another ‘one-and-done’ project.
- Because teams have already collected their data by the time peer evaluations are conducted, and because each team typically analyzes different airline routes, there is little concern about direct plagiarism or copying of tables, figures or analysis. This feature, together with using Turnitin, provides a fairly solid firewall to plagiarism and allows me to focus my grading energies elsewhere.
Some additional background information
The structure of the assignment is fairly typical:
- Week 3: Teams of three or four are formed and provided a preview copy of the project. They are told that we will cover the essential theories they will need in week 5 of the class.
- Week 5: We cover the topic of price discrimination. At this time teams begin collecting data on ticket prices for airline routes with which they are familiar (because students typically come from different geographic regions, it is common for each team to pick different routes).
- Week 6: Teams turn in their projects. At that point, each student is assigned two peer team projects, given a grading rubric, and asked to review the projects while providing constructive feedback (defined as at least three well-thought-out written comments of at least 2–3 sentences each making specific recommendations for improvement). Students are given points for completing peer evaluations in a timely manner. Their grade is based less on the specific content of their suggestions and more on meeting the minimum number and length of written comments.
- Week 7: Peer evaluations are completed. The teams then have 5 days to read and incorporate their peer evaluations before turning in a final report, which I grade using the same rubric.
