National Mentoring Summit January 2013 Agenda Mentor Video Program Description Research Overview Key Findings Conclusions and Continual Improvement Lessons Learned: Mentoring for Academic Gains Action Planning
Video: Be a Mentor http://www.youtube.com/watch?v=YyK_03DccSY Program Model Afterschool, Summer, High School Placement 650 hours/year, + 1000 hours in school Weekly academic mentoring Accelerated lessons, college trips, career exposure Positive culture and youth development approach 2% Only of
nonprofits have this gold-standard randomized research RCT Study Design $4 million over 7 years Over 3 years (2006, 2007, 2008), 951 students & families apply and interview for 3 cohorts 50% accepted through lottery 50% assigned to control group
Researchers test & survey: 4 summers and 3 school years + one 9th grade follow up Two-year results and Summer Snapshot published in 2011 by P/PV Four-year results published in 2013 By MDRC Randomized Control Trial (RCT) Study: funded by Wallace, WT Grant, Smith Richardson, Spencer, Atlantic Philanthropies, Bank of America
Randomization in Brief 1. Hear about HA at school, neighborhood, etc. 2. Interview 99% - enter lottery. 3. Lottery 50% enter program: treatment Treatment Access to full program may never attend.
Control Never able to access program. But given list of other after-school & summer options Annual testing ($120 payment), surveys, Follow-up from researchers Researchers Dr. Jean Grossman Princeton University, P/PV, MDRC Dr. Leigh Linden
Columbia University, University of Texas- Austin Dr. Carla Herrera P/PV Outcome Measures Collected Hypothesis: - With high dosage and structure, Higher Achievement would have a measurable impact on academic outcomes and high school placement, by first impacting attitudes and behaviors. 2 key outcomes: Standardized Test Scores Application, Acceptance to, and Matriculation at Competitive High Schools Other outcomes of interest: Behavior Academic Attitudes
Perceptions of Peer and Adult Support Participation in HA and Other OST Programs Engagement in Academic Activities and High School-Related Activities KEY FINDINGS Two-year follow-up study Higher Achievements intensive year-round program had a significant impact on youths standardized reading and math test scores. The longstanding Higher Achievement model is making a difference in the academic lives of motivated, at-risk students who could easily fall through the
cracks. Summer 2010 Snapshot High levels of attendance and retention, during tricky middle school years: 97% of summer participants also in after-school No summer learning loss. Also - no impact on test scores, compared to control group, over the course of one summer: 2010 Four-year follow up study Math gains persist at
significantly higher levels than the control group. Reading gains level with control group. Significant impact on certain types of family engagement in education Significant impact on placement in competitive private schools and avoiding low-performing high schools Academic Impacts First- Second- FourthYear FU Year FU Year FU Math problem-solving 0.03 0.10* 0.11* Reading comprehension
* = Statistical significance. 0.02 0.08 0.04 Academic and Enrichment Activities Activity Community service First-Year FU Treatmt T-C Diff (%) (%) 53 0
Second-Year FU Fourth-Year FU Treatmt T-C Diff Treatmt T-C Diff (%) (%) (%) (%) 60 4 74 4 Presented ideas to a group out of school Visited a college 63 6
64 9* 69 11* 72 28** 73 28** 78 25**
Read books out of school Writing out of school Visited a business 74 73 56 3 7* 4 79 74 61 4 3
8* 85 82 70 8 14** 14** Events with OST 77 10** 80 6
84 9* Academic contests at OST 68 13** 68 11** 71 16**
Conclusions Year-round, multi-year: high dosage yields results No effect after 1 year, only after 2 years: Academic results take time Reading gains level with control group in 4-year follow-up: Control and treatment made gains. Reading can be more self-directed. Math requires more instruction. Continual Improvement Common Core Standards Alignment: Curricula and Technology Explicitly teach writing skills Staff training to improve inference skills
for reading comprehension How Children Succeed & Social/Emotional Skills Piloted three assessments: PEAR HSA, SAYO, Gallup Student Poll + Grit Scale Annual Operating Plan and Individual Work Plans, tied to Strategic Plan. Annual Retreat: Outcomes and RCT Results Continuous Improvement
Cycle Mid-Year Retreats: Org & City course corrections Update individual work plans Monthly Dashboard and Site Observation Discussions: National & Local Staff School Partners Quarterly Dashboard, linked to Financial Model Maximizing Mentors
Orientation Ongoing mini-trainings Scripted lessons Nightly session feedback Monthly observations One Mentors Perspective Ms. Liesa Her involvement, academic gains of scholars Mentor training and ongoing support Best practices in academic mentoring Table Activity Think, Pair, Share!
What are the goals for your mentoring program? What challenges do you have to meeting your goals? What resources do you have to meet your goals? What resources do you have to overcome your challenges? What support and/or trainings do you have to put in place to maximize your resources and meet the goals of your mentoring program? ACTION PLANNING "Whatever affects one directly, affects all indirectly. I can never be what I ought to be until you are what you ought to be. - Dr. Martin Luther King
