Item analysis

Within psychometrics, Item analysis refers to statistical methods used for selecting test items for inclusion in a psychological test. The concept goes back at least to Guildford (1936). The process of item analysis varies depending on the psychometric model. For example, classical test theory or the Rasch model call for different procedures. In all cases, however, the purpose of item analysis is to produce a relatively short list of items (that is, questions to be included in an interview or questionnaire) that constitute a pure but comprehensive test of one or a few psychological constructs.

To carry out the analysis, a large pool of candidate items, all of which show some degree of face validity, are given to a large sample of participants who are representative of the target population. Ideally, there should be at least 10 times as many candidate items as the desired final length of the test, and several times more people in the sample than there are items in the pool. Researchers apply a variety of statistical procedures to the responses to eliminate unsatisfactory items. For example, under classical test theory, researcher discard items if the answers:

• Show little variation within the sample
• Are strongly correlated with one or more other items
• Weakly correlate with the totality of the remaining items, reflected in an increase in Cronbach's alpha if the item is eliminated from the test

In practical test construction, item analysis is an iterative process, and cannot be entirely automated. The psychometrician's judgement is required to determine whether the emerging set of items to be retained constitutes a satisfactory test of the target construct. The three criteria above do not always agree, and a balance must be struck between them in deciding whether or not to include an item.

References

• Guilford, J. P. (1936). Psychometric methods. New York: McGraw-Hill.
• Kline, P. (1986). A handbook of test construction. London: Methuen