Secrets Revealed: Where Do The NCEES Questions Come From?
There are quite a few mysterious aspects of the NCEES exams, including where the National Council of Examiners for Engineering and Surveying actually gets the questions for the exams. Several questions probably come to mind.
Are they textbook questions? Do they come from engineering professors or textbook authors? Do engineers even write the questions? Who checked the answers? Who made sure they were appropriate for the exam? How do they make sure that the questions represent real-world engineering skills?
This article aims to answer those questions and more.
Test Bank
With the advent of computer-based testing for the NCEES exams, a massive question bank was developed. That means the process for developing full exams has changed. While this bank replaces the professional engineers who used to choose the questions, the process for developing exam questions has remained the same.
When an examinee takes the exam, questions are drawn from the test bank, making it practically impossible for two examinees to take the exact same test. This provides another benefit: it’s harder for people to cheat. They will never know which questions are on the test.
The Test is Reviewed by Professional Engineers
The multiple-choice questions you see come from licensed professional engineers and engineering professors. This reviewing process involves quite a bit of work.
They look at things like whether the question is appropriate for the exam, how long it takes to solve it, whether the problem statement is clear and complete, if the necessary equations are in the reference handbook, and, most importantly, whether or not the answer is correct.
Volunteers
You might be wondering how much people get paid for developing these questions. The answer may surprise you: nothing. All of this is done on a volunteer basis.
The licensed engineers who review all the questions and create most of them are brought onsite to NCEES headquarters in Clemson, South Carolina for a massive meeting.
They are provided with food and lodging, and their traveling expenses are paid. This usually takes two full days, and according to the NCEES these meetings happen several times a year. Sometimes they put out specific calls on the NCEES website for volunteers.
One example of that occurred in 2012, after the decision to computerize the exam. Usually only licensed professional engineers are accepted as volunteers, but an exception was made in this case because it was considered a standard-setting study.
This meant that the goal of the meeting was to develop a revised standard for the test because it was being given through a new medium.
The requirement was that the volunteer had taken and passed the exams, with a preference for those who recently passed them or recently obtained licensure.
The goal was to review and rate the difficulty of existing questions that were to be used for the computer-based version. Their travel and lodging expenses were also paid.
Fairness
The NCEES works hard to keep the question development process free of unfair advantages or conflicts of interest. First, you have to be a licensed professional engineer to review questions for the exam, and to attend one of the question development meetings.
This means that the final decisions on all questions are made by engineers who have not only passed the exams themselves but met all the licensure requirements for their state.
Next, if you are teaching an review course of any kind you will not be accepted as a volunteer. If this policy were not in place, exam questions could easily, and even inadvertently, be leaked. This prevents unfair advantages and conflicts of interest.
Finally, when a question is reviewed it’s not just one engineer that makes the decision but a team of engineers working together. This helps keep the exam questions in a particular section from the influence of a few individuals.
For example, in the heat transfer section, you don’t have to worry that most of the questions will cover just one topic, like convection. A group approach to reviewing questions means that the questions are more likely to represent a broad range of topics, and problem types.
Conclusion
When you take an NCEES exam, you can be confident that the questions are agreed upon by a group of licensed professional engineers from your discipline.
They deemed the questions to be appropriate, doable, and correct. They developed them to prevent one specific area from dominating the exam.
This rigorous process of question development not only makes the exam more meaningful for you, but makes a passing score more valuable in the workforce.
Sources:
National Council of Examiners for Engineering and Surveying