Evaluator.in
Automated submission evaluation — a brief introduction
A little background
At the German University in Cairo Java is used extensively in our CS courses especially during the first 4 semesters. One of the programming courses that just about every engineering major and BI student takes is CSEN 401 Computer Programming Lab. It’s purpose is to teach the students the concepts of Object Oriented programming by asking them to implement a game. Well two actually the first a small text based one and the second a 2D one with a larger scope.
Evaluation is based on JUnit tests, split into two suites a public one whose source is a available to students and a private one that only instructors have access to.
Since the students don’t have access to all the test suites and teachers change them somewhat regularly during the semester students are usually provided no feedback opportunities until after milestone deadlines when the grades are posted. The Students then request office hours from teachers in order to understand why their code is failing. Since all they see is a numeric grade.
On the other side of the coin. Teachers spend a tremendous amount of time downloading, grouping, and manually testing each and every submission of a course that has an average of about 1000 students.
These factors combined have left the course with a particular stigma of being really difficult and time consuming, on both the student body and the faculty.
I am rarely happier than when spending an entire day programming my computer to perform automatically a task that would otherwise take me a good ten seconds to do by hand. — Douglas Adams
This is where my project, Evaluator comes in. An open source project which aims to automate the evaluation of submissions. Never being able to resist the urge to automate! Drawing inspiration from online judges such as SPOJ and uVA
Submissions are evaluated in real time and the results shown, students get instant feedback regarding both private and public suites. However they can only access the source code of the public suites. The test names however provide the information they need to improve their grades. Which are provided along with the standard JUnit ‘X expected but Y found’ message.
The difference between instance feedback and traditional post deadline grading is that the student is motivated to improve his/her grade since they can re-submit and get a better grade without any penalties!
About a year ago Evaluator was created with the intent of increasing the value students receive from courses at the GUC and free teachers up from administrative tasks and help them focus on teaching.
Today it’s my thesis topic and I need to write up a proper paper.
This is the first post in a series about evaluator, the various technologies underlining it and my design decisions. The hope is to further improve educational technologies.
