Evaluation Support 2.0Sponsored by the - Evaluative Thinking

Evaluation Support 2.0Sponsored by the - Evaluative Thinking

Program Evaluation Essentials Evaluation Support 2.0 Session 3 Anita M. Baker, Ed.D. Evaluation Services Bruner Foundation Rochester, New York Evaluation Support 2.0 Sponsored by the Bruner Foundation www.evaluativethinking.org and Evaluation Services www.evaluationservices.co Free evaluation training and technical assistance focused on development of evaluative capacity including data analysis

and reporting. Four (4), on-site, hands-on training sessions. Introduction to and use of free/low-cost tools to facilitate data entry, management and analysis. Guided evaluation project required. Virtual conference with funder, other organization participants.

Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services E-Surveys Key Decisions Why use an e-survey rather than a hard-copy survey/ intercept survey/ alternative survey or other data collection strategy? What question types do you need? How will they be displayed? Do you need an other field? Should they be required? Bruner Foundation Rochester, New York

Anita M. Baker, Evaluation Services 1 Before You Create an Esurvey Strong items Administration plan Analysis plan Respondent experience Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 2 Multiple Choice (only 1 answer)

Forced Choice Item (MO) Directions read: Mark One unless it is so obvious that is the expectation. Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 3 Multiple Choice (multiple answers) Multiple Response Item (MATA) Multiple response items often create analysis challenges. Use

sparingly. Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 4 Comment/Essay Box/Openended Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 5

Matrix of Choices (1 answer/row vs. multi answers/row) Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 6 Likert/Rating Scale Not at all important Slightly important Somewhat Moderately

important important Extremely important Quality of ingredients Flavor Texture Brand A true Likert scale has 5 answer choices, and by the way it is pronounced Lick ert not Like ert. (The strategy was named for Rensis Likert who invented or popularized them.) Bruner Foundation Rochester, New York

Anita M. Baker, Evaluation Services 7 Single Vs. Multiple Textboxes You must have an analysis plan for using these data. Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 8 Numerical Textboxes

You must have an analysis plan for using these data. Consider hard codes (e.g., 1/month, at least 6 times per year, etc.) and using single text boxes; or true numbers when they exist which can then be used in other calculations like averages. Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 9 Printing a Survey From the design survey menu select Print Survey Bruner Foundation Rochester, New York

Anita M. Baker, Evaluation Services 10a Printing Surveys Then Click here This will generate a Survey in PDF format. Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services

10b Evaluative Thinking Survey Response Collector Types Types of collectors: Weblink; Email Tracker; Facebook/Website Post; Manual Data Entry* Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 11

Weblink Collector Write your own cover email Dear Colleague, Recently you attended an Allied Against Violence training session provided by the Center for Anti-Violence Education and agreed to provide some follow-up feedback regarding your use of the materials and strategies. Please click on the following link to start the very brief follow-up survey. It should only take you a few minutes to respond and your answers will be kept confidential. All responses will be combined and summarized to inform CAEs continued work in this field. Thank you in advance for your assistance CLICK HERE https://www.surveymonkey.com/s.aspx Thank you pages (default or custom) Bruner Foundation

Rochester, New York Anita M. Baker, Evaluation Services 12 Follow-Up and Track Responses Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 13 Manual Data Entry: Two Strategies Data entry/use as a database

Weblink Collector: (Fast/Easy) Advanced Settings: Allow more than 1 respondent per computer Survey End Page: Loop to start Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 14 Analyzing Data Using Survey Monkey Key Skills Generating and using frequencies Bruner Foundation Rochester, New York

Anita M. Baker, Evaluation Services 15 Analyzing Data Using Survey Monkey Key Skills Generating and customizing graphics Exporting graphs and tables Bruner Foundation Rochester, New York

Anita M. Baker, Evaluation Services 16 Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 17 Analyzing Data Using Survey Monkey Key Skills Using Filters allows you to select data, remove missing data and use numerical analysis functions.

Comparing data calculates cross-tabs Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 18 Analyzing Data Using Survey Monkey Key Skills Using Filters: Filter by Q&A Bruner Foundation Rochester, New York

Anita M. Baker, Evaluation Services 19 Using Compare Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 20 Compare/Cross-tab Considerations Decide whether to use row or column

percents to present your data. The calculations answer different questions. Do you want to know for each grade group how many rated the program positively (i.e., outstanding or very good)? OR Do you want to know what grade the positive respondents were in? TIP: Pick a strategy and use it consistently. Bruner Foundation Rochester, New York

Anita M. Baker, Evaluation Services 21 Analyzing Data Using SM Exporting and Merging Data Export All - All Summary Data (PDF which can be customized) Export All - All Response Data (Excel or SPSS, Zipped File - sheet 1 has data) Merging Data connecting data from a survey monkey file with other data using a common key Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services

22 Survey Result Example: Excel Summary BEC Session 2 - January 2012 How would you rate Session 2 overall? Answer Options Not So Good Okay Very Good Excellent Response Percent 0 15% 71%

15% Response Count 0 5 24 5 answered question skipped question 34 1 How would you rate Session 2 overall? Not So Good

Okay Very Good Excellent Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 23 SURVEY RESULT EXAMPLE Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services

24 Evaluative Thinking Definitive Statements Percent of Training Participants (N=93) who Think AAV Helped or Will Help Them: Some A Lot TOTAL Discuss issues of violence with clients Access additional strategies for self-care/stress reduction

45% 47% 55% 51% 100% 98% Provide positive interventions for clients 32% 65% 97% Understand the importance of self-care/stress reduction

38% 58% 96% 31% 54% 45% 39% 67% 43% 52% 58% 98%

97% 97% 97% Target = 50% or more say a lot to each Offer clients new ways to: De-escalate Situations Manage Anger Do safety planning Conduct Bystander Interventions AAV met its targets for all areas of interest except Management of Anger More than 95% of participants thought AAV helped or will help them with each of the focus areas of the training. Bruner Foundation Rochester, New York

Anita M. Baker, Evaluation Services 25 Definitive Statements % of 2005-06 Freshman who . .. Reported struggling to maintain grades Peer Study Group Total Yes n=212

No n=257 N=479 36% 58% 47% Are planning to enroll for the Only about 1/3 of freshman in peer study groups reported struggling to sophomore year at this school 89% 73%

80% maintain their grades compared to of those not in study groups. Proportionately more study group participants are planning to enroll for sophomore year. Bruner Foundation Rochester, New York Anita M. Baker, Evaluation Services 26 Survey Monkey Questions? http://help.surveymonkey.com/ Bruner Foundation Rochester, New York

Anita M. Baker, Evaluation Services 27

Recently Viewed Presentations

  • How Accurate Are Self-Assessments of Second Language Proficiency?

    How Accurate Are Self-Assessments of Second Language Proficiency?

    Might there be a more accurate wayto do self ratings? Lately I've been offering a different type of self assessment to students who are wondering what their level of proficiency might be. It tries to improve the accuracy of self...
  • Presentazione standard di PowerPoint

    Presentazione standard di PowerPoint

    0-5' 4 h. da 48 h in poi. lesioni a: - cervello - midollo - cuore - aorta. emorragia cerebrale. emotorace. pneumotorace. fratture pelviche. fratture ossa lunghe
  • Master Sun Consulting Balanced Scorecard - Tool for

    Master Sun Consulting Balanced Scorecard - Tool for

    Strategic Learning Loop Initiatives & Programs test the hypotheses Output (Results) reporting Management Control Loop funding Input (Resources) update the strategy PERFORMANCE 85% of management teams spend less than one hour per month on strategy issues 92% of organizations do...
  • Rénovation du BTS maintenance industrielle

    Rénovation du BTS maintenance industrielle

    niv. BTS. Maintenance des systèmes de production. Maintenance des systèmes éoliens. Maintenance des systèmes énergétiques et fluidiques. Dominique PETRELLA - IA-IPR STI. La maintenance des systèmes de production. Le métier s'exerce principalement dans les entreprises faisant appel à ...
  • Geography Jeopardy Recognize Vocabula Where on Who am

    Geography Jeopardy Recognize Vocabula Where on Who am

    Map key Compass rose symbol This is another way to say "half of the Earth." Globe- half sphere hemisphere North America is located in this hemisphere. Northern Eastern Southern These are what the four directions are called. Cardinal directions North...
  • Lunch 'n Learn

    Lunch 'n Learn

    The Boomerang Principle applies internally(to oneself) and externally(to everyone around you). Internally - As a man thinks, so he is (negative thoughts, self-deprecation, beating oneself up, etc.) Develop a positive attitude. Develop a growth mindset.
  • Lock Out Tag Out (LOTO) Supervisors responsibilities >

    Lock Out Tag Out (LOTO) Supervisors responsibilities >

    Review LOTO. You must complete and document the review if: Employees are observed not following LOTO procedures. Not locking out equipment, tagging out equipment when a locking device is available, have no procedure for the system, etc.
  • BTS530: Major Project Planning and Design

    BTS530: Major Project Planning and Design

    Refactoring Structured, disciplined Rewrite/restructure existing code without changing its external behaviour Apply small transformation steps combined with re-executing test steps Refactoring Why? remove duplicate code improve clarity shorten long methods remove "hard coded" constraints "smelly code" -consider refactoring "code ...