Synthetic Aperture Personality Assessment

Synthetic Aperture Personality Assessment (SAPA) is a method used for telemetric assessment of individual differences, primarily in the context of online surveys. The SAPA method uses data collected from the administration of large inventories of personality assessment items to large pools of participants, though it differs from traditional data collection methods in that each participant responds to only a small subset of all available items. In other words, each participant receives a random (or partially random) subset of the items under study. As long as some of the items are overlapping between pairs of participants, the smaller subset is more palatable for individual participants yet can be combined to synthesize large covariance matrices (with considerable data missing at random). In this way, the SAPA methodology is well-suited for assessing personality and individual differences across multiple domains. It is also a highly efficient means for new item prototyping and scale construction.

The SAPA Methodology

The procedure is straightforward. From a large set of personality and ability items (P), a smaller subset of items (n) are presented to any one subject. With random sampling of the items, all possible pairs of items are eventually presented together. As the number of subjects (N) grows, each item has been given to N × (n / P) subjects, and each pair of items has been given to N × (n / P)² subjects. The online survey developed by William Revelle (and maintained by the Personality, Motivation, & Cognition Lab at Northwestern University) has used this technique with approximately 200,000 participants (N) as of 2012. In this example, each participant receives a subset of items (n) equal to approximately 75, though the full set of items (P) being administered at any given time may be as high as 500.

The Traditional Method Used for Internet Surveys

Studies of individual differences in cognitive and non-cognitive aspects of personality are frequently limited by the sample sizes available in the typical university research setting. Small but stable relationships are difficult to detect when one is limited to 50 to 100 subjects, and detecting complex relationships between multiple measures is difficult when participants are limited to short one or two-hour studies. Alternative procedures involve large research groups collecting data across many research sites (e.g., the Programme for International Student Assessment - PISA). Since the 1990s, an increasing number of psychologists have begun to employ web-based data collection techniques as a means of increasing both the size and breadth of samples with little loss of validity (Fraley, 2004; Gosling, Vazire, Srivastava, & John, 2004; Skitka & Sargis, 2006). While several online surveys have collected data from very large samples (e.g., the >300,000 reported by Gosling et al., 2004), most of these studies administer short questionnaires or basic cognitive tasks (Greenwald, Nosek, & Banaji, 2003).

Explanation of the "Synthetic-Aperture" Analogy

As a variation of standard web-based assessment methods, the SAPA methodology borrows by analogy a technique used in radio and optical astronomy: Synthetic Aperture Measurement. The resolution of a telescope is limited by its diameter which may be functionally increased a great deal by combining input from multiple, linked sites into one coherent image. Effectively, a very large telescope is created by synthesizing the input of many smaller ones. A classic example in radio astronomy is the Very Large Array, part of the National Radio Astronomy Observatory in Socorro, New Mexico, where 27 relatively small (≈ 25 meter) radio telescopes are spread out in a Y-shaped configuration to simulate the resolution of a 36 km telescope. The configuration is adjustable so that the telescope can either emphasize resolution (by maximizing the distance covered) or sensitivity (by concentrating the telescopes close to each other). In optical astronomy similar techniques are used in inferometry at the W. M. Keck Observatory in Hawaii with "outriggers" to supplement the main telescope. Similar techniques are available for "telemetric assessment" of psychological constructs. Rather than combining signals from the same source using different telescopes as is done in astronomy, the structure of personality can be studied by combining the responses of many people across more items than any one person is willing to take. This is not an entirely novel procedure. The Educational Testing Service, for example, has long used the very large samples available when students take the SAT or GRE to develop new items by randomly giving small subsets of items to much smaller (but still quite large) subsamples of students. The SAPA methodology allows for these techniques to be used by a broader population of researchers by making use of open source and public domain software.

Public-Domain Item Pools

During the past century, the measurement of personality and ability has tended to be fragmented by separate groups of individuals using proprietary sets of measures. Indeed, the proprietary nature is partly seen in the choice of names for these inventories and tests: the Minnesota Multiphasic Personality Inventory, the California Psychological Inventory, the Eysenck Personality Questionnaire, the Freiburger Persönlichkeitsinventar, the Guilford-Zimmerman Temperament Survey, the Hogan Personality Inventory, the Jackson Personality Research Form, the Myers-Briggs Type Indicator, the Stanford–Binet Intelligence Scales, the Wechsler Adult Intelligence Scale, the Wechsler Intelligence Scale for Children, etc. Each of these tests was carefully developed by research groups and each is protected by copyright. Although some groups will allow non-profit use of the measures for minimal cost, this is the exception. Many of these inventories have similar sounding scales, but given the expense, there are a limited number of studies directly comparing the inventories (Grucza & Goldberg, 2007).

An alternative to the proprietary nature of personality measurement is the International Personality Item Pool (IPIP) developed by Lewis Goldberg (1999). Including more than 2,400 items in the form of sentence stems, the IPIP collaboratory has at least 269 scales targeted at everything from achievement striving to vitality/zest and includes public domain scales meant to reflect constructs found in at least 17 commonly used personality inventories. All of the IPIP items and the common personality inventories have been given to the same community sample in Eugene/Springfield, Oregon, and item statistics are freely available from Goldberg and his associates. Some have questioned the open and free use of the IPIP items with respect to the possibility of the public learning to fake personality tests used in employment settings. Others have worried about whether the freedom to select items will lead to fragmentation of constructs rather than the hoped for integration (Goldberg et al., 2006).

Even more proprietary than non-cognitive personality scales and items are measures of intellectual ability. Items and scales are either under copyright or are completely idiosyncratic to particular labs and yet still not openly published. As a result, there are currently few alternatives for researchers in search of validated, open source measures of cognitive ability.

Sample Characteristics

As discussed in reviews of web-based research (Fraley, 2004; Gosling et al., 2004; Skitka & Sargis, 2006), the participants in non-directed online surveys are demographically diverse but not a representative sample of anybody other than those who want to take web-based surveys. Samples tend to be about two-thirds female and have a median age of 25. Many techniques are available for recruiting participants with specific demographic profiles, though these efforts necessarily limit the potential size of the total participant pool.

References

• Gruszka, Aleksandra; Matthews, Gerald; Szymura, Blazej (16 June 2010). Handbook of Individual Differences in Cognition: Attention, Memory, and Executive Control. ISBN 9781441912107.
• https://sapa-project.org/dmc/docs/2011Telemetrics.pdf Archived 2017-08-11 at the Wayback Machine
• Fraley, R. C. (2004). How to conduct behavioral research over the internet: A beginner's guide to HTML and cgi/perl. New York: Guilford Press. ISBN 1-57230-997-0
• Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and using the implicit association test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85 (2), 197–216. doi:10.1037/0022-3514.85.2.197
• Grucza, R. A., & Goldberg, L. R. (2007). The comparative validity of 11 modern personality inventories: Predictions of behavioral acts, informant reports, and clinical indicators. Journal of Personality Assessment, 89 (2), 167–187. doi:10.1080/00223890701468568
• Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality psychology in Europe (Vol. 7, p. 7-28). Tilburg, The Netherlands: Tilburg University Press.
• Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., et al. (2006). The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 40 (1), 84–96. doi:10.1016/j.jrp.2005.08.007
• Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. American Psychologist, 59 (2), 93–104. doi:10.1037/0003-066X.59.2.93
• Leon, M. R., & Revelle, W. (1985). Effects of anxiety on analogical reasoning: A test of three theoretical models. Journal of Personality and Social Psychology, 49 (5), 1302–1315. doi:10.1037/0022-3514.49.5.1302
• Mulholland, T. M., Pellegrino, J. W., & Glaser, R. (1980). Components of geometric analogy solution. Cognitive Psychology, 12 (2), 252–284. doi:10.1016/0010-0285(80)90011-0
• Raven, J. (1989). The Raven Progressive Matrices: A review of national norming studies and ethnic and socioeconomic variation within the United States. Journal of Educational Measurement, 26 (1), 1–16.
• Skitka, L. J., & Sargis, E. G. (2006). The internet as psychological laboratory. Annual Review of Psychology, 57, 529–555. doi:10.1146/annurev.psych.57.102904.190048