Handbook of Learner Evaluation and Test Item Construction
Writing Valid, Reliable Test Items
Do you need to write test questions and want some guidance? Do you have a test you would like to revise?
The Handbook of Learner Evaluation and Test Item Construction—formerly only provided to fellows in the nationally known Primary Care Faculty Development Fellowship Program—is now being made available to CHM faculty and staff.
This handbook has three sections. The first, titled Qualities of All Good Tests, defines the qualities of purposeful, valid, reliable, objective, comprehensive, differentiating, expected, instructive and useful. Two examples are presented for each quality. The reader is asked to identify the example that more closely exemplifies the quality. The correct answer is then identified and explained.
The second section, General Standards for Test Items, explains the standards for test items and provides examples. The standards are:
- There are sufficient test items to cover all the important ideas.
- All test items are on ideas related to instructional objectives.
- Questions simulate the use of the test content as specified in the objective.
- All questions are easy to read.
- There is only one right answer.
- Don't give the answer away.
- The answer to one question doesn't affect the answer to another question.
- Questions are grouped by type.
- Questions refer to a source if needed.
The final section, Technical Specifications for Types of Test Items, explains how to write different kinds of test items, and includes explanations and clinical and non-clinical examples for each. The eight types of test items are:
- True or False
- Multiple Choice
- Matching
- Completion
- Short Answer
- Essay
- Production
- Procedures Checklist
The Appendix to the Handbook provides a worksheet for each type of test item you want to write. These worksheets show a summary of the specifications and provide space to write the related objective and test items, all on one page. The Handbook is a PDF document formatted to be printed double-sided and can be downloaded here.
Advice from the Handbook:
To make the most out of this manual you should have at your side the test items you have already written so you can check them against the guidelines. If your items don’t meet the guidelines, study the qualities, general standards and technical specifications then rewrite the test items accordingly.