In my 11 year career as an instructor, I’ve had to pass a lot of certification exams. In many cases not on the first try. Sometimes for fair reasons, and sometimes, it feels, for unfair reasons. Recently I had to take the venerable CCNA exam again. For various reasons I’d allowed it to expire, and hadn’t taken many exams for a while. But recently I needed to re-certify with it which reminded me of the whole process.
Having taken so many exams (50+ in the past 11 years) I’ve developed some opinions on the style and content of exams.
In particular, I’ve identified some types of questions I utterly loath for their lack of aptitude measurement, uselessness, and overall jackassery. Plus, a couple of styles that I like.
This criticisms is for all certification exams, from various vendors, and not limited to even IT.
To Certify, Or Not To Certify
The question of the usefulness of certification is not new.
One one hand, you have a need to weed out the know-its from the know-it-nots, a way to effectively measure a person’s aptitude in a given subject. A certification exam, in its purest form, is meant to probe the knowledge of the applicant.
On the other hand, you have an army of test-dumping dullards, passing exams and unable to explain even basic concepts. That results in a cat-and-mouse game between the exam creators and the dump sites.
And mixed in, you have a barrage of badly formed questions that are more appropriate to your local pub’s trivia night than it is a professional aptitude measurement.
So in this article I’m going to discuss the type of questions I despise. Not just because they’re hard, but because I can’t see how they accurately or fairly judge a person’s aptitude.
Note: I made all of these questions up. As far as I know, they do not appear on any certification exam from any vendor. This is not a test-dump.
The story goes that Albert Einstein was once asked how many feet are in a mile. His response was this: “I don’t know, why should I fill my brain with facts I can find in two minutes in any standard reference book?”
I really relate to Einstein here (we’re practically twinsies). So many exam questions I’ve sat through were pure pedantic trivia. The knowledge of the answer had no bearing on the aptitude of the applicant.
Here’s an example, similar to ones I recall on various exams:
What is the order of ink cartridges in your printer? Choose one.
A: Black, Blue, Cyan, Yellow
B: Yellow, Cyan, Blue, Black
C: Blue, Cyan, Black, Yellow
Assuming you have a printer with color cartridges, can you remember the order they go in? Do you care? Does it matter? Chances are there’s a diagram to tell you were to put them.
Some facts are so obscure they’re not worth knowing. That’s why reference sources are there.
I can even make the argument about certain details about regularly used aspects of your job. Take VRRP for example. For network administrators, VRRP and similar are a way to have two or more routers available to answer to a single IP address, increasing availability. This is a fundamental networking concept, one that any network administrator should know.
VRRP uses a concept known as a vMAC. This is a MAC address that sits with the floating IP address, together making a virtual router that can move between physical routers.
So far, everything about what I’ve described about VRRP (and much more that I haven’t) would be fair game for test questions. But a question that I think is useless is the following:
The vMAC for VRRP is (where XX is the virtual router ID):
I’m willing to bet that if you ask 10 good CCIEs what the vMAC address of a VRRP is, none would be able to recite. Knowledge of this address has no bearing on your ability to administer a network. How VRRP works is important to understand, but this minutia is useless.
I have two theories where these questions come from.
Theory 1: I’ve written test questions (for chapter review, I don’t think I’ve written actual certification questions) and I know it’s difficult to come up with good questions. Test banks are often in the hundreds, and it can be a slog to make enough. Trivia questions are easy to come up with and easy to verify.
Theory 2: Test dumpers. In the cat and mouse game between test writers and test dumpers, vendors might feel the need to up the difficulty level because pass rates get too high (which I think only hurts the honest people).
Another one I really despise is when a question asks you for the exact command to do something. For example:
Which command will send the contents of one directory to a remote server using SSH?
A: tar -cvf directory | ssh [email protected] “cd /home/user/ tar -cvf -”
B: tar -xvf directory | ssh [email protected] “cd /home/user/ tar -xvf -”
C: tar -cvf directory > ssh [email protected] “cd /home/user/ tar -cvf -”
D: ssh [email protected] “cd /home/user/ tar -xvf -” > tar -xvf directory
For common tasks, such as deleting files, that’s probably fair game (though not terribly useful). Most CLIs (IOS, Bash, PowerShell) has tab completions, help, etc., so that any command syntax can be looked up. Complex pipes like the former are the kind I use with some regularity, but I often have to look it up.
The Unclear Questions
I see these in certification tests all the time. It’ll be a question like the following:
What are some of the benefits of a pleasant, warm, sunny day? (Choose Three)
- A: Vitamin D from sunlight
- B: Ability to have a picnic in a park
- C: No need for adverse weather clothing
- D: Generally improves most people’s disposition
Look at those answers. You could make an argument for any of the four, though the question is looking for three. They’re all pretty correct. Reasonable people, even intelligent, experienced people, can disagree on that correct answer is.
Questions I Do Like
I try not to complain about something if I don’t have something positive to contribute. So here’s my contribution: These are test questions that I think are more than fair. If I don’t know the answers to these types of questions, I deserve, in every sense of fairness, to get the question wrong.
A scenario question is something like this: “Given X, what would happen”.
For example, if a BDPU was received on portfast enabled interface, what would happen?
If a host with an IP netmask combo of 192.168.1.10/24 was to try to communicate with a host configured on the same Layer 2 segment with an IP address of 192.168.1.119/25, would they be able to communicate?
I like those types of questions because they test your understanding of how things work. That’s far more important for
There are some network basics, that might seem like trivia, but knowing would be important to know. For example:
What is the order of a TCP handshake?
A: ACK, SYN/ACK, SYN
B: SYN, SYN/ACK, ACK
C: SYN, ACK/SYN, SYN
D: ACK, ACK/SYN, SYN
This question is fundamental to the operations of networks, and I would hope any respectable network engineer would know this. This would be important for TCP dump analysis, and other fundamental troubleshooting.
Runner Up For Questions I Hate: Non Proof-Read Questions
At one point the quality of a certain vendor dipped quite a bit. So much so there were obvious, glaring errors in them. One test question was something like:
What are the benefits of technology X?
The options had zero to do with technology X. Fortunately, the answer was obvious. The question was asking about technology Y, which happened to start with the same letter as technology X. So I’m pretty sure I got that question right. I made a note in the exam, as you’re allowed to do that.
I took the next version of the test, and the same question was there, with the same mistake from at least a year prior.
Another one was a questions that just stopped mid-sentence. Something like:
When doing X with Y, what is the
That was it. It was like someone was writing a question, got up to get some coffee, and then went right to the options. Again, on that question it was enough to get what they were asking, and I’m pretty sure I got the question correct.
But come on. We’re paying hundreds of dollars for these exams.
If you write test questions, ask yourself: Would the best people doing what this question tests get this answer right? Is it overly pedantic? Is there a clear answer?
This was mostly written as a frustration piece. But I think I’m not alone in this frustration.
from The Data Center Overlords https://datacenteroverlords.com/2019/03/12/certification-exam-questions-that-i-despise/