|
|
|
|
Home > Certification > How Exams are Developed > An Insider's Look at Item Writing An Insider's Look at Item Writing
The Item Writer's Challenge
 |
All item writers are trained to write valid test questions, or items. Rarely do they know how challenged they will be when they are accepted onto an item writing panel. The introduction in their item writer's handbook, however, quickly gives them insight:
The Human Resource Certification Institute's (HRCI) exams are only as good as the individual items from which they are comprised. These items must be valid, reliable and able to discriminate between those who have mastered human resource knowledge requirements and those who do not possess such mastery. ( Item Developer’s Handbook , p. 1-2).
During training, emphasis is placed the importance of writing and formatting items to be at the appropriate cognitive level for the appropriate exam.
HRCI classifies all items according to the following levels:
-
Knowledge/Comprehension: Recalling factual material, translation or interpretation of a concept.
-
Application/Problem Solving: Applying familiar principles or generalization to solve real-life problems.
-
Synthesis/Evaluation: Combining distantly related elements into a whole involving critical judgments in terms of accuracy or consistency of logic.
Item writers are challenged to develop items that:
- Are relevant and appropriate for an HR generalist practitioner to know.
- For the PHR and SPHR exams, differentiate between the levels in terms of format and cognitive level necessary to correctly answer the items.
- Are of varying difficulty levels.
- Are of an appropriate reading level.
- Have distractors or foils (incorrect choices) that are plausible yet wrong.
Item reviewers are challenged to determine if the items they receive to review:
- Are appropriate for the HR generalist to know.
- Are correctly and concisely stated and premised correctly.
- Are grammatically correct.
- Pose a correct answer that is clearly the best answer.
- Pose distractors that are plausible but not the best answer.
- Are current.
- Are culturally, ethnically and geographically fair.
- Are clearly written at the appropriate level.
- Are coded to the HRCI test specifications.
In item review, items can be minimally edited or substantially rewritten. Some items that cannot be improved upon will be rejected.
After each test administration, performance data on each item and the exam form as a whole are carefully evaluated. There are a number of statistical measures routinely calculated for each item and reviewed. Items that do not perform to established standards are either discarded or placed back in item review for further revision. This continuous item review ensures that exams are both valid and reliable.
Why Multiple-Choice Items?
 |
HRCI uses multiple-choice items on exams because they:
- Are flexible and adaptable and enable measurement of a wide range of knowledge, skills and abilities.
- Tend to be more reliable than most other exam formats.
- Can be written to cover a single content area at a variety of difficulty levels.
- Allow for a wide sampling of knowledge, skills and abilities in one exam because of their brevity.
- Can be machine scored.
- Can assess the ability to integrate information from several sources.
- Allow for more sophisticated statistical item analysis and overall test analysis procedures.
Multiple-choice items consist of three parts, the premise (or stem), the correct answer and distractors. The premise states the problem or the question to be answered. The correct answer is one of the four options that represent the correct response or the best correct response. In this case, "best" means that a panel of experts would agree to this judgment. Distractors are the incorrect responses. A question is considered to be good if the distractors are plausible yet wrong or not the best possible option.
The Premise (Stem)
 |
Test takers can expect two types of premise statements when taking an exam; a statement posing a question or as an incomplete statement (for example, "Which type of psychological test measures a person's overall ability to learn?" and "A person's overall ability to learn is best measured by a[n]...").
There are other types of premise statement categories. Among the most common are purpose ("What was the primary reason for passage of the Immigration Reform and Control Act [1986]?"), definition ("The fact that a test actually measures what it is supposed to measure is called..."), ordering ("What is the first step in validating a test?"), cause and effect ("The comparable worth issue is caused by which compensation practice?"), recognition of error ("Which interview question would bring a disability discrimination charge?"), association ("One possible by-product of extensive job simplification is...") and similarity ("The EEOC and the OFCCP both investigate...").
Item writers are taught to write items that measure meaningful and relevant knowledge requirements that reflect the knowledge or skills actually needed in the performance of an HR generalist's job duties. Items should require knowledge that a good candidate will possess and a poor candidate is not likely to have.
The Correct Answer and Distractors
As difficult as it is to write a good premise to a test question, it can be just as difficult to write the correct answer and distractors. Item writers are taught the distinction between items in which only one response is correct and items in which more than one response may be technically correct. They are urged to write items that force the candidate to choose the "best answer." For candidates, it may appear that more than one choice is the correct one. The questions are specifically designed that way to assess the candidate's knowledge of HR. Few actual work experiences present one possible answer; indeed, HR generalists must wade through a variety of possible solutions and choose the best possible one on a daily basis. The items candidates find on the HRCI certification exams, then, must assess not just the ability to recall facts, but the ability to apply them as well.
Scenario Items
Scenario items present an HR situation. A number of questions follow based upon the information presented in the scenario. Scenario items require candidates to integrate knowledge requirements from many different areas of the HR body of knowledge and are particularly well-suited for SPHR candidates. They often present typical situations that senior-level HR practitioners encounter and test the diverse knowledge required to resolve those situations.
|
|