Encouraged by ChatGPT’s ability to mark and feedback on student responses, by contrasting them against a given mark scheme (see here), I decided to come up with a low-code solution that teachers could easily use to mark multiple responses, for multiple questions.
The Plan
The strategy here is a simple one — define a custom function in Google Sheets that allows the user to send prompts to GPT and retrieve the output. Each prompt would be a concatenation of the mark scheme and the student response, and the output would be the score and GPT’s feedback.
Step-by-Step Tutorial
Step 1: You will need an OpenAI API key for Google Sheets to ‘talk’ to GPT. If you are a new user, you will get $18 worth of free credit, but they do expire after a while. If your free credits have expired (like mine, sadly), you will need to set up a paid account. That sounds scary, but they charge only the tokens used (which is infinitely better than a recurring subscription). For all my (quick and dirty) experimentation, I have used a grand total of … $0.05 worth of tokens. I did some quick calculator-punching, to auto-mark a class assignment comprising 10 short-structured questions, for 40 students, it should cost about $ 0.30.
To go to the OpenAI webpage with your API keys, click on the circular icon on the top right-hand corner of the page. Never share your API key publicly, otherwise, someone out there could be happily chatting with GPT, at your expense.
Step 2: Open a Google Sheet, go to Extensions, followed by Apps Script.
Step 3: Delete the existing code within:
and paste the following code which defines the function GPT3PROMPT:
const API_KEY = "YOUR API KEY HERE";
function GPT3PROMPT(prompt, temperature = 0, tokens = 2000) {
const url = "https://api.openai.com/v1/chat/completions";
const payload = {
model: "gpt-3.5-turbo",
messages: [{"role":"user",'content': prompt}],
temperature: temperature,
max_tokens: tokens
};
const options = {
contentType: "application/json",
headers: { Authorization: "Bearer " + API_KEY },
payload: JSON.stringify(payload),
};
const res = JSON.parse(UrlFetchApp.fetch(url, options).getContentText());
return res.choices[0]['message']['content'];
}You will need to enter your API key above in the first line. You can also rename the function if you wish to. I will discuss more about the temperature and tokens at the end.
Step 4: Click on the Save icon, then the Run icon.
Google will seek your authorization for your custom function to access the OpenAI model (i.e., GPT 3.5 Turbo for the example above). Follow the prompts to grant access.
If you get an error message at the end of ‘Run’, do not worry. To check, return to the Google Sheet, go to a cell and type:
=GPT3PROMPT("Hi")
Press Enter. You will see within the cell Loading…, followed by GPT’s reply in a short while.
Congratulations! You have defined your own custom function to ‘chat’ with GPT in Google Sheets.
Step 5: In this step, we will set up the mark scheme in a new tab. When we send the prompts to GPT to mark later, I will reference the cell containing the relevant mark scheme.
Here’s mine, the mark scheme for 3 different questions (i.e., cells B2, B3 and B4):
Step 6: To mark the student responses, you can copy the student responses into one column in the main tab. In the neighboring cell, type the following:
=GPT3PROMPT(CONCATENATE("Based on the mark scheme - ",'Mark Scheme'!$B$2," Score the student response: ",B4," Reply in the form: (Score / 2), Feedback"),0,2000)Here we are using the custom function defined in Steps 1–4 to send a prompt to GPT. The prompt is telling GPT: Based on the mark scheme found in the tab ‘Mark Scheme’, cell B2, score the student response found in cell B4 of the current tab. Reply in the form: (Score / 2), followed by the feedback.
If you have different questions, you just have to change the cell referenced in the formula (i.e., $B$2) to the relevant cell (e.g., $B$3 for my 2nd question, or $B$4 for my 3rd question).
Step 7: Select the cell with GPT’s response and drag down to auto-mark the remaining responses. Voila! That’s it!
Here’s a clip showing Steps 6 & 7
Conclusion
GPT continued to perform well in terms of its marking accuracy and feedback provided. Some tweaking to the mark scheme is needed e.g., alternative answers to accept / reject, or necessary key words for the mark to be awarded.
I will not delve into the details of the GPT’s marking accuracy here but I have discussed the results in greater detail previously (see here).
The easiest way for teachers to auto-mark using GPT would be to copy and paste the mark scheme and student responses into ChatGPT. However, there are some disadvantages to this method. The first would be that your input into ChatGPT would be limited by the token length (4096 tokens or about 3000 words). Also, when the prompt is overly long, ChatGPT might start to hallucinate. Lastly, having to copy and paste into ChatGPT the mark scheme and responses, and go through the same processes when extracting the marks and feedback from ChatGPT’s interface, is for me, an unwieldy solution.
The steps presented here are low-code, low-cost, and provide a neater solution since student responses from learner management systems and Google forms can be readily downloaded as a .csv or .xlsx file. The model used is also an off-the-shelf one, and can easily be replaced with a better model e.g., GPT4 .
Each student response is sent to GPT as a prompt, together with the mark scheme. Hence, for short-structured questions, it is unlikely you would cross the 4,096 token limit. In the code for the custom Google function, you can also set a limit to the number of tokens used for the prompts.
To conclude, do screen through the eventual marks and feedback. GPT’s output is non-deterministic (it may change its response when posed the same question over time). One way to make GPT’s response more predictable is to set the temperature in the custom function code to 0. So, here’s one other benefit of the method here, the user has more control over GPT’s settings as opposed to ChatGPT.
