Design of Quantitative Survey Instruments

Design of Quantitative Survey Instruments

Design of Quantitative Survey Instruments ScWk 242 Session 6 Review Questions What are the 10 guidelines for conducting fieldwork? What are your responsibilities while taking fieldnotes? Content Analysis 1. Purpose of Content Analysis Make inferences to the source of communications Understand the motives, values or intensions of those who wrote them Understand the symbolic meaning/communication 2. Diaries/personal documents Letters to the editor

3. Inferences to populations Used to ascribe the values of the audiences intended for the communication to reach (the demographic) Movie content 4. Evaluate the effects of communications Treatment effectiveness Notes, video of behavior/interactions (reduction in behavior based upon certain types of responses by therapist to client 5. Structured observations Use with other data collection methods Content analysis of observational data can specify what is observed and what measurements are to be used Content Analysis

1. Process of Content Analysis a. Systematic Follow the same procedures to examine each occurrence Reliable, can be replicated by someone else 2. Objective Ensure validity and avoid bias Rules to count an observation must be impartial Define the criteria to be applied Make explicit the rules to be applied to control for any special interest or ideology that might impact analysis Content Analysis Steps in Content Analysis Develop research question Establish what you want to study Ask the appropriate questions

Select a Sample What is the sampling strategy? What is the available universe? How best to represent the population? Which has the best content? What is the best cohort? Content Analysis 1. Select the Unit of Analysis Specify what you will count Words, phrases, or concepts Establish complex categorical systems coding rules (pictures of minorities) procedures 2. Coding, Tallying and Data Analysis Coding is dependent upon the unit of analysis Gender: 2 categories, male female Grade Level Completed: 20

Construct a tally sheet do the counts and frequencies Content Analysis 1. Latent vs. Manifest Content Latent content is an indicator of the underlying meaning of what is communicated. Manifest content is specific, objective and easy to record. Developments during the review of manifest content recurring themes keep reviewing the content look for similarities look for what makes this consistent 2. Have another rater use same criteria for coding the manifest content to establish inter-rater reliability 3.

Report the results Questionnaire Construction Review General Format (space, space & more space) Contingency Questions Relevant to only some respondents e.g. male or female Matrix Format Visually pleasing caution: some questions are not amenable to this format, encourages a response bias

Question Ordering Check boxes are best or circling number next to a response Can elicit bias Order has impact, random ordering does not help (previously practiced) Be sensitive, only defense Instructions, section instructions Pretest/Pilot test Composite Measures Review Item Selection (face validity)

Unidimensionality (deal with same construct) Variance (differing response by different people) Reliability Validity Avoid bias: + or -, social desirability, cultural bias, etc. Scaling procedures Likert Scaling

Semantic differential (choose between opposites using words e.g. interesting vs. boring with Likert scale in between) Guttman Scale (harsh attitude embedded in other questions) Thurstone Scales (panel of judges with every conceivable question on the subject/construct and analyze for inter item agreement) Question Formulation Knowledge, Opinions, Attitudes & Beliefs Think through and research as to what might be the best answer for the knowledge area of investigation National surveys track public health efforts of government agencies (AIDS, heart disease, smoking) Review literature and previously used questionnaires and studies to use what has already been tested and for comparison of your work to these

other studies. Phrase questions as attitude or opinion questions rather than knowledge questions (e.g. in your opinion, what do you thing increases the likelihood of cancer? A) lack of exercise Sleeper questions/responses e.g. use a fictional company or program that if respondent says yes they have heard of it, then you can code their responses as inaccurate, keep track of this rate for comparison to chance Final Questionnaire/ Survey Development Considerations Use methods/devices that have worked in the past Where possible, validate aggregate estimates using other data sources Conduct pilot studies Ask respondents which questions were especially

threatening Errors in Questionnaire Development Under & Over reporting errors Telescoping or including events from outside the time period in question Omit events that should have been included Individuals remember salient events (hospitalization, anniversaries, deaths) Memory Aids Records e.g. checkbooks, appointment books/PDAs, hospital bills or other records. Consult prior to interview

Diary, ask respondents to record in diary (longitudinal studies), calendar/PDA, day off work, prescription filled, etc. Problems: burdensome to respondents, if using diaries inherent underreporting over time. Food intake diaries Errors in Questionnaire Development If the question has a socially desirable response, it is threatening (e.g. study habits of students, sex, eating, reading the newspaper, TV watching, internet porn, etc.) Use common words, but not the vernacular. If your respondent uses them you are not advised to repeat the terms. Respond with Thats another way to say it. Phrases to facilitate accurate reporting: Load the question before hand so that individuals do not feel that their response is deviant or outside the norm.

Authority citation (health issues) Approve of behavior in advance (many people have different religious practices, some are considered unusual such as burning incense or making offerings, but all are based in firm beliefs about religion. What are your religious practices? Assume individual engages in behavior. In the past 2 weeks how many cigarettes did you smoke? (can be offensive to those who do not engage in those behaviors) Errors in Questionnaire Development Sentences Sometimes longer sentences increase the likelihood of an accurate response by as much as 25-30%. Provides fuller explanation of what is being asked and allows individual time to think about what is being asked. Responses

Open ended responses can capture information that is traditionally underreported. Response ranges can be intimidating if they do not include the frequency at which the individual uses substances. How many? allows the respondent to identify the frequency. Errors in Questionnaire Development Instructions Questionnaire Design Build in transitions and introductions into sensitive topics to warn

the individual and give them the opportunity to decide how to respond. Ask about past behavior then proceed to current behavior Ask sensitive questions at the end of the interview Questionnaire Administration (field procedures) Anonymous methods Self report questionnaire Randomized response technique (coin toss to determine which question to answer, interviewer does not know the result of coin toss). Determine rate of responses to see if consistent with probability of coin toss.

Recently Viewed Presentations

  • Geography - Yukon Conservation Society

    Geography - Yukon Conservation Society

    Preferential rights for the Inuvialuit in harvesting. Under the Inuvialuit Final Agreement, Inuvialuit have. The preferential right to harvest all species of wildlife, except some birds throughout the Yukon North Slope(IFA, 12 (24) (a))
  • 2016 PROCUREMENT FRAMEWORK Standard Procurement Documents PRIVATE SECTOR

    2016 PROCUREMENT FRAMEWORK Standard Procurement Documents PRIVATE SECTOR

    Re-bidding as a last measure. Cost and Quality weighting system in RFPs. VfM = the effective, efficient, and economic use of resources, which requires the evaluation of relevant costs and benefits, along with an assessment of risks, and non-price attributes...
  • Combs Ford Surgery

    Combs Ford Surgery

    COMBS FORD SURGERY Patient Information Screen October 2015 Book your Seasonal Flu Vaccination now! STAY SAFE THIS WINTER Free vaccinations if you are; Over 65 A Carer Pregnant Or suffer from; Heart problems or a chest complaint Asthma requiring regular...
  • Data persistence -

    Data persistence -

    Programmatic vs. Interactive Database Access. DB. Program. DB Driver. ... Book Club Entity Relationship Diagram (ERD) Book Club Database Model (UML) Modeling Inheritance Relationships. How do we map the following Class Model to an RDBMS.
  • Hex Quiz

    Hex Quiz

    a plague of hornets. God sent a plague of flies. What was the fifth plague God sent on Egypt? all their livestock died. mad cow disease. a plague of lions. A plague of crocodiles. All their livestock died. What was...
  • Representing nitrogen, phosphorus, and carbon interactions in the

    Representing nitrogen, phosphorus, and carbon interactions in the

    Compared with predecessor versions, ELMv1-ECA better predicts global-scale gross primary productivity, ecosystem respiration, leaf area index, vegetation biomass, soil carbon stocks, evapotranspiration, N. 2 O emissions, and NO 3 - leaching. Figure1.
  • #HeyScientistMel


    This is the 5th episode of Hey scientist mel. A podcast where I talk about all of the science things. I also take your questions and topics thru twitter, email, and FB. Speaking science by revealing relevance, boosting literacy, and...
  • Higher quality. lives. Healthier HIPAA Privacy & Security

    Higher quality. lives. Healthier HIPAA Privacy & Security

    Always use cover sheet when faxing. ... PHI "at rest" is stored on desktops, laptops, servers, mobile devices, USB flash drives, CD's & DVD's, etc. PHI "in motion" is moving across a network, including wireless transmissions ... Risk analysis/assessment. Information...