Talk:Questionnaire construction/Archives/2012

Some notes - prepared for intended upgrade of article

Definition

Surveys are a social research method .. systematically collecting data about the same variables from a number of dfferent cases :

"..asking the same questions across a broad group of people in order to obtain comparable information from them"

Surveys are conducted to acquire data of a kind and quantity sufficient to describe the characteristics of whole sets of cases

As Questionairres are the easiest way to systematically collect this kind of data, they are the most common technique used in survey research

History

1086:

William the Conqueror surveyed the wealth and landholdings of England using a standard set of inquiries and compiled the results in the “Domesday Book.”

mid 1930's:

survey research is an industry .. first developing in it's modern form in the mid 1930's, when quota sampling was first applied to voting behaviour studies .. and to determine characteristics of radio listeners, magazine readers, and product purchasers .. enabling people and organisations (including advertisers) to better and more profitably target their services and/or messages

1950's:

Japan developed a survey research industry soon after World War II

1980's:

India, South Korea, Jamica, Greece, Mexico (and many others) had all developed their own survey research capabilities (within universities and/or provate sector)

By the 1980's, in United States, survey research had developed into a major industry employing approx 50 000 people..

Theory

19320000:

Renis Likert introduced what is now the most commonly used form of scaling .. being a 5 point scale eg "would you say you agree a lot with the statement, agree a little, are neutral, disagree a little or disagree a lot with each statement?".

Likert did more than introduce the format. He developed techniques of tease apart, obtain data, and measure the various dimensions of what are otherwise multidimensional subjects. His method was to take a long list of possible scaling items for a concept, and find the subsets that measured the various dimensions. If the concept was unidimensional, then one subset would do. If it were multidimensional, then several subsets were needed.

The steps for building a Likert scale are as follows:

• identify and label the variable you want to measure - this is an exercise in induction from experience;

• write a long list of indicator questions or statements - this is an exercise in induction from experience;

• determine the type and number of response categories. Some popular response categores are agree-diagree, favour-oppose, helpful-not helpful, many - none, like me - not like me, true-untrue, suitable - unsuitable, always - never. Most Likert scales have odd number of responses ie 3, 5, 7, giving people a range of choices that include a midpoint;

19780000DILM:

Don Dillman, of the Survey Research Laboratory at Washington State University, synthesized research into response rates for mailed questionnaires, and developed the "Total Design Method" for questionairre surveying, which includes the following

• questionnaire is best in a booklet format of a size sufficient to be sent in an envelope .. of a size and color designed NOT to look like advertising, and not as intimidating as (official?) letter sized paper;

• no questions on either the front or back covers of the questionnaire booklet. Front cover to contain a title that provokes respondent's interest, plus an eye-catchng illustration, and instructions. Back cover should contain note thanking respondents and open-ended question inviting comment on the questionairre;

• pay careful attention to the order of questions, such that the first questions directly relate to the topic of study (as per title on front page), also first questions ought to be interesting and easy to answer; plus non-threatening. Once someone starts a questionairre they are likely to finish it, so more threatening/private questions can be introduced later, but not clustered together..

• construct the pages of the questionnaire according to standard conventions. Captial letters for instructions to respondes & mixed upper and lower case for the questions themselves. Never allow a question to break at the end of a page and continue on another page. Use plenty of paper; don't make the instrument appear cramped.

• keep self-administered/mailed questionairres down to 10 to 12 pages, with no more than 125 questions. Beyond that, response rates drop.

• send out the questionnarie with a one-page cover letter - explaining in the briefest possible terms, the nature of the study, how the respondent was selected, who should fill out the questionairre (respondent or members of the household etc), who is funding the survey, and why it is important for the respondent to send back the questionairre. The cover letter must also guarantee confidentiality & explain the presence of any id number. These cover letter MUST be personal/ individualised, and each signed personally with a blue point pen (leaves verifiable indent!)

• package the questionanairre, cover letter, and reply envelope in an envelope with names typed on to the front (and if to be mailed pack, include colourful, commemorative stamps on reply envelopes)

• pay careful attention to contact procedures. Send letter to each respondent explaining the survey and informing them that a questionnaire will be coming along soon. Send a post card reminder our to all potential respondents a week after sending out the questionairre. Send a second cover letter and questionairre to everyone who has not responded 2 weeks later. Finally, 4 weeks later, send anouther cover letter and questionarrire, along with an additional note explaining you have not received the respondents questionairre, stating how important it is that they participate in the study.

19790000SCPR

Schuman and Presser attempted to determine whether forced choice (aka closed) questions or open-ended questions are better. They sdid this by conducting a survey asking people about their reasons for chosing their particular job, using forced choice with some respondents, and open-ended with others.

• Schuman and Presser found significant disparities in the kinds of descriptions one might give of peoples reasons, depending on whether their choices were forced or open ended .. with the forced choice clearly focusing attention and concentrating responses to the limited range of choices given, relative to open-ended questioning (ie 17% vs 2% chosing job for "chances for advancement"; & 59% vs 21% chosing job because it is "important work".)

1983BRAD

Bradburn, reviewing surveys done in industrialised nations, found that when questions are threatening (such as questions about masturbation, alchol consumption, or drug use).. responds are 50 to 100% more likely to report such things where questions are open-ended vs forced choice (closed) questions.

Bruceanthro (talk) 14:39, 2 June 2008 (UTC)

Kinds of Questionairre

There are three methods of collecting survey questionnaire data:

Personal, Face-to-Face Interviews

Questionnaires are administered Face-to-Face - such that you have some capacity to explain questions and to some extent control against responds misunderstanding or misinterpreting questions

Self-administered questionnaires

Questionnairs are completed independently of the researcher, often in 'the privacy of one's own home', 'at one's own leisure', and anonymously - such that recepients often need some incentive to participate, and there is no opportunity to explain questions and control against misunderstanding

Telephone interviews

Telephone interviewing is an increasingly significant method of gathering survey data, particularly in countries where very high percentages of the population are able to be contacted by telephone - and telephone interviews combines the advantages of a face to face interview with the advantages of a self-administered questionnaire being a more impersonal and removed from the respondent.

Testing & Measuring Concepts

Concepts are ideas people create to more easily or readily understand things, often referred to by one or more terms that people create to more effeciently communicate their understanding. They are a simple, shorthand tool summarising often abstract, and complex sets of behaviour, attitudes or characteristics which people believe share something in common .

Underlying any survey, and, any questionnaire therefore are some core concepts which each refer, as a kind of shorthand, to more abstract complex sets of behaviour, attitudes or characteristics ..about which the people administering the questionannaire are seeking to obtain data. .

'It is up to us to first define what we mean by the concept an dthen develop indicators for the concept as it has been defined. By their very nature definitions are neither true nor false: they are only more useful or less useful"

Bruceanthro (talk) 03:38, 2 June 2008 (UTC)

Questionnairre design

Question formats - open-ended, closed-ended, multiple choice, and rating scales.

Open-ended questions - should be simple, seeking clear, short answers eg, "What is your birthdate", or "How many times have you been to hospital this year"

Closed-ended questions - should be concise, with a complete listing of mutually exclusive response categories.

Rating Scales - usually appropriate only for a literate and/or better educated peopulation/ sample

Question order -

broad -> specific - questions should proceed from broad, general requests for information to those requesting detailed, more specific information - so requests for detailed info doesn't bias requests for more general information information.

less personal -> more personal - less personal questions shold proceed those that seem to be questions likely to be perceived as more private or threatening.

Comparative questions- if you want to compare your sample to a wider sample, use some questions from a large population survey (eg national census)

Bibliography

• Bernard, H. Russell (1995) (2nd Ed) Research Methods in Anthropology: Qualitative and Quantitative Approaches. Altamira Press, London. ISBN 0-8039-5245-7.

• Dillman, D.A (1978) Mail an dtelephone surveys: The ttal design method. MacMillan, New York.

• Vaus, D.A de (1985) Surveys in Social Research. Allen & Unwin. Sydney.ISBN 0-86861-623-0

• Jenkins, Cleo R & Dillman, Don A (1995) "Towards a Theory of Self Administered Questionairre Design" in L. Lyberg, P. Biemer, M. Collins, E. DeLeeuw, C. Dippo, N. Schwarz, and D. Trewin (Eds) Survey Measurement and Process Quality. Wiley-Interscience. New York. Accessed from United States Census Bureau Website, 1 June 2008

• Martin, Elizabeth (2006) "Survey Questionnaire Construction" Research Report Series. Accessed from United States Census Bureau Website, 1 June 2008

• Weller, Susan C (1998) "Structured Interviewing and Questionnaire Construction" in H. Russell Bernard (Ed) Handbook of Methods in Cultural Anthropology. Altamira Press, London. ISBN 0-7619-9151-4

References

Bruceanthro (talk) 13:30, 2 June 2008 (UTC)

Shouldn't this be merged into 'Questionnaire'? (The current content on the Questionnaire page is pretty vapid.) Brianpie (talk) 07:56, 23 September 2008 (UTC)

Proposing move of section

I am propsing to move the table in the subsection headed "Methods of collection" to the recent article Survey data collection on the grouynd of bringing that material together, and because it has little to do with "Questionnaire construction". Comments? . Melcombe (talk) 16:54, 6 July 2011 (UTC)