By Saaz Rai Under CISSP Dec 18, 2017

CISSP Computerized Adaptive Testing

Effective 18th December 2017, CISSP exam candidates will be taking the Computerized Adaptive Testing (CAT). The term "adaptive" indicates that every exam is tailored to match the candidate's ability.

In the previous linear exam format, every candidate was required to answer 250 questions within a maximum duration of 6 hours. With the new CAT format a candidate would have to answer anywhere between 100-150 questions depenidng on the candidate's ability.

Should you be worried? How does this affect your CISSP exam preparation? Is there a change in the exam content or the question type? Do you need to take a CAT simluation practice in order to pass the exam?
Here I'll try answer some of these questions which many of you may be contemplating.

Why did (ISC)2 change the exam format to CAT?

There are several benefits in moving to a CAT format:

  • The exam duration is much shorter (less than half). This could mean more $ savings for (ISC)2 which I reckon would be in millions.
  • CAT provides a more precise and efficient evaluation of a candidate’s competency. This is a well proven system for testing being used by many other exams like GMAT etc.

Has the exam content or the question type changed?

The answer to both questions is NO.
The new CISSP CBK will be applicabe only from April 2018. Until then there is no change in the exam content.

There is no change in the question type and it will still include scenario based questions & MCQ, Drag & Drop and Hot Spot questions.
Both exam formats use the same content, the same question formats, the same domain weights and the same passing standard.

How does Computerized Adaptive Testing (CAT) work?

The first item presented to the candidate will be a medium level difficulty question. If the candidate is able to answer the question correctly the difficulty level of the next question will be raised. However, if the answer is incorrect, the difficulty level of the next question will be reduced.

Each candidate is presented with a minimum of 100 questions and a maximum of 150 questions. Through this iterative process, the computer is to gauage the true ability of the candidate's ability and accordingly pass or fail the candidate.

Is the CAT exam format more difficult than the previous linear format?

The difficulty level of each question presented to the candidate is dependent on the candidate's ability. Lets look at an example of Alice, Bob and Charlie. Alice's ability is higher than the passing standard and she would have easily passed the exam even in the linear format. On the other hand Bob's ability is below the passing standard. While Charlie has prepared well for the exam his chances of passing the exam is dependent on how well he performs during the examination.

Alice is likely to get a question with a medium level difficulty to begin with which she is likely to answer correctly. The next question for Alice will be relatively difficult than the previous question with a 50% probability of Alice answering it correctly. There might be few questions which Alice is unable to answer to correctly but mostly Alice will answer correctly. Its very unlikely she will be presented with an easy question and she will feel that questions are challenging and matching her ability. After Alice completes answering 100 questions, the system will determine with over 95% statistical confidence that Alice's ability is above the passing standard. Result: PASS!

Bob is also likely to get a question with a medium level difficulty to begin with which he might answer incorrectly. The next question for Bob will be relatively easier than the previous question. Most questions which Bob faces is well below the passing standard. Even though he might answer the easier questions correctly he is unable to answer questions with value over the passing standard and is thus left to answer only the easier ones. After Bob completes answering 100 questions, the system will determine with over 95% statistical confidence that Bob's ability is below the passing standard. Result: FAIL!

Similarly Charlie is likely to get a question with a medium level difficulty to begin with which he might answer correctly. The next question for Charlie will be relatively difficult than the previous question. Charlie is likely to answer many questions correctly. After Charlie completes answering 100 questions, the system is still unable to determine Charlie's ability. In this case Charlie is presented with a maximum of 150 questions. If, for the last seventy-five (75) questions, Charlie’s ability estimate is consistently above the passing standard, for each and every item, then the exam result is a PASS. If, at any point over those last seventy-five (75) questions, Charlie’s ability estimate falls below the passing standard, the result is a FAIL.

Because the exam is adaptive, all three Alice, Bob and Charlie will think that most questions asked were challenging.

How does CAT affect your exam preparation?

CAT should have zero affect on your preparation. As mentioned earlir, CAT uses the same content, the same question formats, the same domain weights and the same passing standard.

How does CAT affect my exam experience?

The CAT format will be a different experience than the previous linear exam format.

  • The exam duration is significantly lower with a minimum of 100 questions to a maximum of 150 questions. The maximum duration allowed is 3 hours. You should be able to complete the exam within 2-3 hours.
  • There is no possibility to mark a question for review. In the linear format one could mark an question for review and revisit the question at a alter time.
  • A difficult question is a good sign that you are performing well and vice-versa.

Do you need to take a CAT simluation practice test?

Again the answer is NO. The exam will test your knowledge and ability and the exam format or the question type shouldn't really matter at all.

It would be very difficult to prepare a CAT simulated test since the algorithm used by (ISC)2 is unknown. However, you can still use the linear test to practice your knowledge though you should reduce the number of questions to 150 and test duration to 3 hours. Avoid "Mark for review" option.