Reliability Validity Levels of Measurement Scales How we figure out what to measure Conceptualization Process of taking a construct and refining it by giving it a conceptual or theoretical definition Research focusing on college students In Ohio? What region? Age? Major?
Operationalization Links a conceptual definition to a specific set of measurement techniques Coming up with a measure
Remember the conceptual definition Keep an open mind Borrow from others Anticipate difficulties Dont forget units of analysis Empirical Hypothesis The degree of association How well
operationalized variables are associated (or not) with the concept construct determines the hypothesis Reliability Reliability means dependability or consistency Same thing occurs over and over under same conditions
A scale, for example How dependable is the study? Is the study consistent, or does it yield wide varying results? Can the study be replicated? Reliability Measurement directly affects the quality of conclusions. Care is needed to make sure that results are not corrupted by improper measurement.
The operational definition of a concept should have a precise meaning: The terms by which you measure a concept should be explicit. Reliability Reliability and validity are the biggest threats to proper measurement. Reliability is the extent to which an experiment, test, or any measuring procedure yields the same results on repeated trials.
Do you get the same result every time? Reliability Three tests of reliability: Test-retest method Applying the same test to the same observations after a period of time and then comparing the results of the different measurements Alternative form method Two different measures of the same concept
administered to the same respondents at different times before the scores are compared Split-halves method Divide a multi-item measure into two measures with both of the new measures applied at the same time Improving Reliability Clearly conceptualize constructs Increase level of measurement Use multiple indicators of a variable
Triangulation Use pretests, pilot studies, and replication Validity A valid measure is one that measures what it is supposed to measure, in other words, the degree of correspondence between the measure and the concept it is thought to measure. Four tests of validity
Validity Truthfulness Refers to the match between a construct and a measure Want it to be valid for a particular purpose and definition How good is the measure? Is the data measured correctly?
Is the data analyzed correctly (statistical)? Internal Validity Are there errors as a result of the internal design of the study? Are there errors as a result of the controls? Internal validity problems can occur from a flawed survey along with a multitude of other factors
External Validity Can your experiments findings be generalized? External Validity questions are evident in every study; however, methods exist to keep external validity high and the number of external flaws low Types of Validity Face validity
Judgment that the indicator really measures the construct Content validity Does your measure represent the full content of a defintion? Criterion validity Use some standard or criteria to indicate a construct accurately Concurrent validity Indicator must be associated with a preexisting indicator judged to be valid
Predictive validity Indicator predicts future events that are logically related to a construct Types of Validity
1) If I create a new test of mathematical ability for high school students and test it by having high school math teachers look at it and tell me if it seems appropriate, I am measuring for ____________________ validity.
2) If I am examining an individuals ability to cope with stress and have three attributes I am particularly interested in and I am checking to see if my construct hits on all three attributes, I am measuring for _______________________ validity. 3) If I create a new test for cognitive recognition and students that score high on it also score high on previously existing tests for cognitive recognition, I have demonstrated ____________________ validity. 4) If I compare my measure for testing the potential to suffer from childhood diabetes with a previously used test, I am looking for _________________________validity. 5) If I create a new test of intelligence and students that score high on it also do better in college than those who score lowly, I have shown ___________________ validity.
Validity Tests of validity are not as good as tests of reliability. Reliability is easy to demonstrate through some form of repeated trials. Validity is more difficult because we can never be sure about the true value of a concept: Especially true with abstract concepts Validity
Whereas a valid measure is reliable (because if truly valid, it will measure the concept correctly every time), a reliable measure is not necessarily valid. The measure could be measuring the concept incorrectly in a consistent way. Relationship between reliability and validity Levels of Measurement
The level of measurement of a variable describes The amount of precision associated with a variable The mathematical properties of the variable Both precision and mathematical properties increase as you increase the level of measurement from nominal to ratio. Levels of Measurement
Continuous v. discrete variables Continuous Have an infinite number of values or attributes that flow directly along a continuum Temperature, age, income, crime rate Discrete Relatively fixed set of separate values or attributes Gender, religion, marital status Nominal Level
Only reports a difference Candidate preference, religious preference, Yes/No, etc. Discrete Variables Levels of Measurement The level of measurement of a variable describes
The amount of precision associated with a variable The mathematical properties of the variable Both precision and mathematical properties increase as you increase the level of measurement from nominal to ratio. Ordinal Level Rank ordered Grades, opinion
Strongly Agree Agree Disagree Strongly Disagree
Levels of Measurement At the ordinal level, categories may be ranked in order in addition to indicating a difference between categories. Example: Please indicate the highest level of education you reached (elem., high, college, more). Precision: A little more precision and can be used with more statistical tools Interval/Ratio Level
A specified distance Interval does not contain a true zero point (ratio does) Interval: IQ, SAT Ratio: years of school, income Levels of Measurement The interval level includes all of the information of the preceding levels and
adds meaningful intervals between values of the variable but does not use a meaningful zero. Example: What did you score on the SAT? Precision: More precision and can be used with most statistical tools Levels of Measurement The ratio level adds a meaningful zero to the interval level. Example: How many years of education?
Precision: Most precision and can be used with most statistical tools Scales Some concepts can be captured with a single question. More complex concepts may require a multi-item measure consisting of several questions that capture different components of the concept and increase validity.
Scales Summation index: Combines the scores on multiple questions to create one single measure of a concept Likert scale: Uses only select questions from an index that differentiate between different respondents to create a single score for each respondent
Scales Guttman scale: Has answer choices arranged in an ordinal manner; respondents will agree with each of the lower-ranked answers if they agree with a higher-ranked answer Factor analysis: Allows researchers to uncover patterns across related measures to create summary variables that represent different dimensions of the same concept
Mutually Exclusive One and only. One may only fit the criteria of one category Ex: Religion: Christian, non-Christian, Jewish, Buddhist NOT MUTUALLY EXCLUSIVE Exhaustive All cases fit into one category Ex: If the Election were today, would you
support Sherrod Brown, the Democrat, or Mike DeWine, the Republican or neither? NOT EXHAUSTIVE Missing Data No survey is perfect, and certain questions will be left unanswered or completely skipped Remedy? A catch all category and/or a way to factor out missing data Yet, missing data can still mislead a study
One Last Thing Feeling Thermometers Likert Scales Response set problem How do we fix this???
Attend to the novel/unusual See physical attributes Drawn to the non-threatening See behavior next Draw Inferences about person STEP 2: Deciding to Talk Will they find me attractive STEP 3: Exit or continue Search for similarities STEP 4: Make Behavior...
Scrooge was: "as hard and sharp as flint" and "as solitary as an oyster". Elaboration questions - examples! Why does Dickens use similes at all, rather than just saying he is hard and lonely? Could these similes describe any other...
Five Light: Weston Winn, Jeremy Oakes. Spring 2012. A Quick Reminder. Functional Behavior Analysis. The "ABC's" of behavioral analysis (Antecedent, Behavior, Consequence) App designed to assist parents in recording and organizing data.
Parts of the Caliper The Vernier Caliper The vernier caliper has two main parts which has the scales that the measurements are read from. The main beam:this is the non moving part of the unit and contains the main scales,...
Periodic Table and Chemical Reactions revision The chemical element found in diamonds… Carbon A metal which when burned produces a bright white light… Magnesium An element used to make jewellery… Gold Metals can …
Be identified by a red helmet, yellow safety vest and/or Warden ID card if it is safe to do so. (Visual Aids) Emergency Control Organisation Wardens are identified by a red helmet, yellow safety and/or warden ID card if it...
transitional cell carcinoma of the urothelium is shown here at low power to reveal the frond-like papillary projections of the tumor above the surface to the left. It is differentiated enough to resemble urothelium, but is a mass. No invasion...
Northern Ireland: Derek Reay, Geological Survey of Northern Ireland (GSNI) and Dr. Rob Raine, Geological Survey of Northern Ireland (GSNI) Sarah Blake, Sean Mason, Catherine Buckley and John Burgess of ARUP at the 3rd National Geothermal Association of Ireland Conference...
Ready to download the document? Go ahead and hit continue!