Computer Security Education

I can distinctly recall a day in elementary school when I had a sudden moment of clarity — that homework was a thing that I didn’t have to do. Since that day, I’ve mostly taken my education into my own hands. When I want to learn something, I figure out where the best resources are and I learn it.

This is common in the security industry, where most professionals and experts are self-taught and our best research develops out of a weekend project.

“I have never let my schooling interfere with my education.” — Mark Twain

Security education is a topic that we hear about a lot — from employers not being able to hire anyone (pdf) to the notion that there are going to be 1 million new computer security jobs this year. The only conclusion we can draw from these kinds of articles is that organizations are having a hard time hiring competent computer security professionals.

I suspect the root cause of this problem is not that students and professionals aren’t interested in computer security, but because it is so hard to be sufficiently educated in computer security.

At the bottom of this article, I propose a university curriculum that can educate students despite these issues.

Universities

tl;dr: Universities have bad degrees, bad courses, bad teachers, and provide no way to tell the difference between them.

Students who go to school for computer security (often under Information Assurance, Information Systems, or Cybersecurity Management degrees) are rarely taught the skills they need to succeed. This is because the curriculum is often designed and taught by professors who have never been formally trained in computer security and have never professionally worked in computer security.

While most fields have a similar problem, some have worked around them by influencing curriculum from external organizations. For instance, the American Medical Association requires that students go to an accredited medical school, complete a medical licensing exam, and complete a residency before becoming a doctor. The American Bar Association requires that students pass the Bar exam before becoming a lawyer.

This isn’t the fault of universities — computer security is such a young field that even if we date its birth from the first public record of where memory corruption was theorized to result in arbitrary code execution in 1972, the security field is only 44 years old in 2016. That’s barely long enough for the first generation of security professionals to retire and become professors. Additionally, modern computer security curriculum has only been around for less than a decade — not long enough for universities to receive actionable feedback about the curriculum.

Most students who know they want to study computer security, use the NSA’s Centers of Academic Excellence list of designated institutions to look for potential universities. Unfortunately, the way schools get awarded an NSA CAE designation is the same way PCI audits are done — checklists.

On their website, the NSA has a list of requirements your university degree program must meet to become a center of academic excellence. But these requirements are simply a list of topics to cover that rarely goes into meaningful detail and often references obscure, obsolete documents as textbooks such as The Protection of Information in Computer Systems (Saltzer an Schroeder, 1975) and The Trusted Computer System Evaluation Criteria (The “Orange Book”) from 1985 (both referenced in the NSA CAE IA/CD Knowledge Units document). They even provide you with a spreadsheet you can fill out to make sure that your curriculum covers their arbitrary, ineffective checklist.

It’s not simply that the process is bad, it’s that it’s very difficult to tell different schools apart. Students should be able to identify the strengths and weaknesses of individual schools and compare them in specific concentrations that students want to study. The NSA center of academic excellence designation gives no way for students to do that.

Continuing Education

tl;dr: Certifications and trainings have bad instructors, bad materials, perverse incentives, and provide students no way to figure out which is best for them.

No matter how you feel about computer security certifications, you probably agree that course instructors are not paid fairly, which leads to a constant decrease in quality of instructors. When these courses are hands-on or exercise-based, an instructor who really understands the material is critical so students understand the full context of the skill rather than following a series of steps.

A popular alternative to certifications are conference trainings (like those at BlackHat, Recon, and Infiltrate). This is the hardest of all the things I’ve mentioned so far in which to find faults. Conference trainings are usually the best training you can get, but they are often very expensive (on the order of several thousand dollars for a day or two) and they can vary widely in quality.

Certification and training organizations are often incentivized to overcharge and cut corners. Cheaper instructors means thicker margins. Students are asked to pay for everything on top of the course — textbooks, tests, and worst of all, renewal fees.

Certification and training organizations are also incentivized to not grade students fairly. When certifications or trainings are purchased in bulk by an organization attempting to train its employees, these organizations are incentivized to make sure almost everyone successfully passes the course. This keeps the students and their employers happy and more likely to be repeat customers.

Individuals looking to continue their education are often frustrated when trying to evaluate which course or training is best for them. Only someone who has already taken the program or is an expert in the subject can properly judge applicability and quality, because there is no independent organization that vets courses and trainings, the only way to differentiate these courses is through word of mouth and past reviews — which are often unreliable.

Curriculum

Because there will always be a shortage of great teachers and the relevant specifics that should be taught will always be changing, I suggest teaching computer security like mathematics is taught in the United States — where the science and concepts are taught and treated as important, but the part on which students are tested consists of specific, repeatable skills and exercises.

Students should learn specific skills, like reverse engineering, malware analysis, and vulnerability discovery. Everything else should come later, whether on the job, in higher education, or in some sort of residency.

In this way, everyone leaving school would have a reasonable set of useful skills that can be readily applied in almost every computer security position.

Here are a set of courses that I believe would best serve students in the field (These courses assume computer science competency):

Introduction to Systems — This course will teach students how different large, complex systems work and how to analyze different parts of them. I recommend teaching this course like a survey of systems, introducing students to a wide variety of different large, complex systems over the semester. Here some are potential examples:

  • glibc malloc (implement a memory allocator, idea from CMU Systems course)
  • Linux (kernelspace/userspace privilege separation, process management)
  • Java (or Python, Ruby, JavaScript, or any interpreted language with a virtual machine)
  • uEFI and Secure Boot
  • Sandboxes in Windows, OS X, Linux, ChromeOS, Android, iOS, Google Chrome, Internet Explorer, or Firefox

Source Code Comprehension — This course will teach students how to read code as well as they write it, or better. Students will learn code comprehension tools and techniques. Assignments will be to read thousands to millions of lines of code and to answer specific, detailed questions.

Introduction to Reverse Engineering — This course will introduce students to taking apart a small, closed system and understanding it. Here some are example assignments:

  • A simple file format (Bitmap, Zip, ELF, PDF)
  • A single function in a compiled program
  • A network or file protocol

Introduction to Post-Exploitation — Alternatively called Introduction to Networks, this course will teach students common methods of moving laterally around a network after initial compromise. It will cover topics like network scanning, local escalation of privilege, discovering high-value targets, and data exfiltration.

Introduction to Vulnerabilities — This course will simply introduce students to different kinds of vulnerabilities. It will focus on introducing students to new and different types of vulnerabilities. Students will learn how to identify vulnerabilities and understand their properties. Assignments will include discovering and exploiting vulnerabilities.

Introduction to Malware — This course will introduce students to analyzing and understanding malware. Exercises will be practical and will utilize real malware samples. Students will determine common tactics among different malware families and develop detection/prevention/monitoring techniques for them.

Introduction to Forensics — This course will introduce students to tools and techniques to understand and interpret operating system and application data structures (including file systems, configuration stores, and process memory). Assignments will include performing real investigations and writing custom tools to analyze specific data structures.

A Final Note

I truly believe that you get out of education what you put in. Until better education programs exist, either take your education into your own hands or take your time to find a program that fits your needs (this is an old post where I describe my methodology for picking a university to study security).


Thanks to Evan, Dino, Brad, Kelly, Andrew, Marc, and Justin.

Julian attended the NYU Polytechnic School of Engineering (now NYU Tandon) from 2009 to 2013. At NYU Engineering Julian taught Manual and Automatic Program Analysis in 2014 and a CTF Team Independent Study Class in 2011 designed to get intelligent students to work on practical security problems. Both classes were a huge success with many students open sourcing their projects and many students turning their projects into internships and jobs.

At NYU, Julian was best known for running the school’s ISIS Lab (now rebranded as the OSIRIS Lab) from 2010 to 2013. In his tenure there, he developed their internationally-recognized Hack Night training program, revamped the CSAW CTF competition into the largest CTF competition in the world during the 2012 calendar year, and mentored undergraduate and graduate students in developing practical security research.