OUSPG Computer Security Course, iteration 2 (2018)

After years of lobbying to be able to provide a public course on computer security in the University of Oulu, the Oulu University Secure Programming Group OUSPG finally had a chance to organise one last year. With the experience we gained in Autumn 2017 we set to redesign the course for the Autumn 2018 iteration. That course is now over, and we are happy to report good results and interesting insights.

OUSPG teaching efforts visualised

THE COURSE

The course planning was based on a few basic principles:

• Practicality first: our research group has always had a very practical focus on security, and this needs to show in the course.
• Transparency and predictability in evaluation: students know how the course is graded and can choose to act in a way that results in a grade of their choice.
• Continuous evaluation: grade is formed through individually graded parts throughout the course.
• Students have chosen their field for a reason, and joined this course for a reason. We must not lose sight of this: the students are here to learn and we can expect them to be motivated.
• Teachers should work most with the best students (reasoning behind this and other concepts is topic for another text)

Additionally, we increased the amount of staff working on the course, allocating our new OUSPG researchers with responsibility of developing individual laboratories.

As a result, the course had:

• Lecture series of 7 two-hour lectures supporting the practical laboratories.
• 5 practical laboratory sessions (on weeks 1–5), with optional additional work outside the laboratory times (a fuzzing lab, malware & botnet lab, network lab, shellcode lab and a hardware testing lab with the ChipWhisperer Lite)
• A final project with 4 different difficulty tiers to choose from

The grading was built on 0–25 points for the 5 different topics handled and 0–15 points for the final project. Each week, the students could choose to

• Answer lecture questions to take 1 point (passable) OR
• Attend the laboratory session of 4 hours to take either 2 or 3 points (for satisfactory and good performance respectively) AND…
• …if they wanted, they could complete more exercises during the following week to take 4 or 5 points for the topic (laudable and excellent performance, respectively).

This would give them 0–25 points. Any student could additionally take 2 more points for the lecture questions of weeks 6 and 7. For the final project, they could choose from:

• Not doing it (if they have enough points) for 0 points
• Make a personal risk analysis for 2–4 points
• Write a short research article on a topic they choose, 2–10 points
• Reproduce some technical project on security, 2–13 points
• Complete a technical project that has the potential to create a real contribution to security, 5–15 points

In total, students could attain up to 42 points, with 10 points (7 lecture questionnaires + passable personal risk analysis for 3 pts) enough to pass the course. The students knew from the beginning, that this was an option, and coming to the laboratories was their choice, and consequently participation in the labs was for us a sign of commitment to achieving a higher grade.

RESULTS AND FEEDBACK

Lecture and lab attendance remained high throughout the course, and students worked enthusiastically on the problems that were provided. Some additional themes were requested: for example a mobile phone oriented lab was clearly something that would’ve been well received. While less than 20% of students responded to the final questionnaire, based on those feedbacks and discussions with students, the grading scheme was well liked.

However, we would’ve liked the students to go more for the higher grades: finally just 10 of the students worked enough to take grades 4 and 5, and the non-technical final projects were much more popular than the practical ones, as shown in the grade distribution table below:

Grade distribution divided by student choice of final project tier

As we can see, most of the students that just got the passing grade (1) didn’t return a final project, and doing so would’ve easily given them a higher grade. In a separate questionnaire, most of these students confessed to having been too lazy to make a final project, as they knew they would pass.

So, we can see that about 11% of the students (10) worked their way up to the highest grades (4–5), and slightly less (8%, 7 students) attempted a final project that had the potential of being a real-world contribution to security.

In general the feedback from the course was very good: apart from one dissatisfied student the official course feedback system was filled with praise, and favourable comments, some even calling the course the best they had experienced in the university.

BEST WORKS

Still, several students chose to go for a contribution project (Tier 4), and we’re happy to announce, that some of these works are either already public or are going to be published in the near future. Follow our social channels, we are going to mention them there as they are being made public!

WORK FOR THE FUTURE

Where to from now? The course in 2019 needs to be at least as good, and additionally there is a push to make the course available to a larger audience as a network-enabled course.

• We need to convince students that making a contribution is not necessarily hard — too many students were afraid of choosing the Tier 4 final project, as it was perceived to be too hard.
• In general, the non-technical final projects were in many ways more popular than expected. There should be more clearly defined Tier 3 practical projects available, to encourage students to choose them.
• Lack of basic skills caused a lot of problems for some students, and the laboratory materials could be more helpful to these students to allow them to reach the 2 point level more easily.
• Many of the students receiving higher grades and making good projects still had too little contact surface with the teachers. We need to further the idea, that the better work you do, the more contact you will have with the teachers. After all, these people are the future (and many even current) experts in computer security, and consequently they represent the core audience of courses such as this one.
• Obviously there are changes to be made to the existing laboratories, and lectures need to be updated again for next year.