AchieveNJ: Teacher Evaluation Scoring Guide 2013-14 Overview This presentation provides information on how districts compile evaluation ratings for teachers in AchieveNJ. Each element of the evaluation results in a 1 - 4 rating, which is weighted according to state formulas shown in later slides. Overviews and examples are provided for scoring each of the multiple measures. The presentation concludes with information on using each of the multiple measure ratings to calculate one final summative evaluation score for each teacher. 2 Multiple Measures All teachers are evaluated based on multiple measures. Practice Student Achievement Teacher Practice Based on classroom observations All teachers and principals Student Growth Objective (SGO) Set by teacher and principal Student Growth Percentile (SGP) Based on NJ ASK performance Summative Rating Overall

evaluation score Less than 20 percent of teachers 3 Teacher Practice Scoring Teacher practice is measured according to a district-chosen observation instrument, such as Danielson, Marzano, McREL, etc. (see here for complete list). Local have discretion on how to combine observation data and evidence about a teachers practice collected throughout the year into a final teacher practice rating on a 1 4 scale. The example that follows show how the different components of the teacher practice instrument might be calculated. This is an example, not a recommendation, as districts have discretion in determining these calculations. Please consult your District Evaluation Advisory Committee (DEAC) to inquire how this is being done locally. 4 Teacher Practice: Weighting of Domains and Components Across different elements of each instrument, some districts have identified certain components, standards, or domains that they would like to weight more heavily. Below is an example of how a district might weight different components: Planning 20% Environm ent 30% Instructio n 30% Professional ism

20% Summative Teacher Practice Rating 100% Example (domain score multiplied by the weight): (3.25 x 0.20) (4.0 x 0.30) (3.00 x 0.30) (2.00 x 0.20) 3.15 5 Student Growth Objective (SGO) Scoring SGO scoring can be approached in several ways. The specific approach must be determined at the local level (district or school), and will depend on the approach the individual teacher is taking, the subject that is being taught, and the quality of the assessment being used. In scoring an SGO, the 1 4 rating may be based on how many students included in the SGO met their goal as shown below, although other measures of success may be used in cases of small classes sizes, for example. Objective Attainment Based on Number of Students Achieving Class Size 30 students Target/Growth Score 4 3 2 1 90% (27 students) or more met goal

80% (24 students) or more met goal 70% (21 students) or more met goal Less than 70% (20 or fewer ) met goal 6 SGO Scoring When teachers have 2 SGO scores, these can be averaged to reach a summative SGO rating, in this case, the teacher would receive a 2.5 Example: Objective Attainment Based on # of Students Achieving Target/Growth Score Measuring Progress 4 3 2 1 *90% or more students met goal *80% or more students met goal *70% or more students met the goal *Less than

70% of students met goal Objective Attainment Based on # of Students Achieving Target/Growth Score Measuring Progress 4 3 2 1 *90% or more students met goal *80% or more students met goal *70% or more students met the goal *Less than 70% of students met goal e numbers will be determined by teacher and principal based on knowledge of students to create a rigorous and attainable goal. 7 Tiered General SGO: Physics 1 For some teachers, tiering student goals based on preparedness levels might be the best way to structure an SGO. In this example, in order to reach a final score, the evaluator can take a straight (or weighted) average of the student results in each group. Goal Preparedness Group Low Medium High

75% students will meet their designated target scores on the Physics 1 post-assessment Number of Students in Each Group 36/65 21/65 8/65 Target Score on Post-Assessment (%) 70 80 90 8 Scoring a Tiered SGO The table below shows the results of the tiered SGO from the previous page. This shows how to calculate a weighted score that will fairly represent the learning in groups of different sizes. More detailed information on scoring can be found in the SGO Guidebook (pg. 21). Results of SGO Weight Number of (Number of Students Objective students in Reaching Attainme group/total Target nt Level students) Score Prepared -ness Group Number of Students in Group Low 36/65 0.56 27 3 Medium 21/65

0.32 18 4 High 8/65 0.12 4 2 Total SGO Score Weighted Score 0.56x3 = 2.24 0.32x4 = 0.96 0.12x2 = 0.24 3.25 9 Student Growth Percentile (SGP) Scoring Median Student Growth Percentile (mSGP) scores provided by the Department are translated from a 1 99 into a 1 - 4 score according to the conversion chart below and then used in a summative rating. Example: If a teacher earns an mSGP of 59, he/she will receive a rating of 3.2, as shown below. mSGP Score 1 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 Evaluation

Rating 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 mSGP Score 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 Evaluation Rating 2.5 2.5 2.6 2.6 2.7 2.7 2.8 2.8 2.9 2.9 3.0 3.0 3.0 3.0 3.0 mSGP Score

50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 Evaluation Rating 3.0 3.0 3.0 3.0 3.0 3.0 3.1 3.1 3.2 3.2 3.3 3.3 3.4 3.4 3.4 mSGP Score 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 - 99 Evaluation Rating

3.5 3.5 3.5 3.6 3.6 3.6 3.7 3.7 3.7 3.8 3.8 3.8 3.9 3.9 3.9 4.0 10 SGP Conversion Chart Explained mSGP Score 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 Evaluation Rating 2.5

2.5 2.6 2.6 2.7 2.7 2.8 2.8 2.9 2.9 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.1 3.1 3.2 3.2 3.3 3.3 3.4 3.4 3.4 Why are all the values between 45 and 55 set to the same score (3.0)? The Department believes that educators in the middle of the mSGP distribution are driving significant academic growth in their students. Educators whose students achieve scores in this range should be recognized by receiving a rating on par with their impact. 11 SGP Conversion Chart Explained mSGP Score Evaluation Rating 1 20 1.0

21 22 23 24 25 26 27 28 29 30 31 32 33 34 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 Why are the values at the extreme ends of the distribution, 1-20 = 1 in this case (and 80-99 = 4), set to the same score? When more than half of a teacher's students are in the top 20 percentile points (80-99) on the SGP scale it is an indication of very high growth. When more than half of a teacher's students are in the bottom percentile points (1-20) this is an indicator of low growth mSGP Score 65 66 67 68 69

70 71 72 73 74 75 76 77 78 79 Evaluation Rating 3.5 3.5 3.5 3.6 3.6 3.6 3.7 3.7 3.7 3.8 3.8 3.8 3.9 3.9 3.9 80 - 99 4.0 12 SGP Conversion Chart Explained mSGP Score 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 Evaluation

Rating 3.5 3.5 3.5 3.6 3.6 3.6 3.7 3.7 3.7 3.8 3.8 3.8 3.9 3.9 3.9 80 - 99 4.0 Why Decimals? Why Tenths? The use of decimals instead of whole numbers enables the scale to increase/decrease gradually, improving the statistical efficiency of the conversion. This prevents large rating differences that may not accurately reflect significant differences in student learning. 13 Scoring the Summative Rating This section describes scoring for the final summative rating. Practice Student Achievement Student Growth Objective (SGO) Teacher Practice Based on classroom observations Set by teacher and principal All teachers

and principals Student Growth Percentile (SGP) Based on NJ ASK performance Summative Rating Overall evaluation score Less than 20 percent of teachers 14 Summary of Standard Setting Process Setting Performance Levels Approximately 90 educators worked for three days analyzing data and making contributions to the summative rating scales. Performance Level Descriptor (PLD) meeting: 1 day, 70 educators Summative Scale Setting Meeting: 2 days, 20 educators (both days) Educators examined anonymous teacher portfolios developed based on data from pilot districts. The educators recommended the scale below, which the Partially in full: Highly Department has adopted Ineffective Effective Effective 1.0 1.85 Effective

2.65 3.5 4.0 15 Component Weighting for Non - SGP Teachers For teachers who do not receive an SGP score, the scoring breakdown includes an SGO rating and a teacher practice rating (see image). 2013 2014 Weights: Non-Tested Grades and Subjects These ratings are calculated as individual components on a 1 - 4 scale at the district level and reported to the Department. The following pages include examples of how a summative rating can be reached. Teachers Outside of Grades 4-8, Language Arts Literacy and Mathematics 15% Student Achieveme nt 15.00% 85.00% Student Growth Objectives 85% Teacher Practice Teacher Practice 16 Summative Rating Example (Non

SGP Teacher) Example 1: Highly Effective Teacher Component Raw Score Weight Weighted Score Teacher Practice 3.60 0.85 3.06 Student Growth Objective 3.75 0.15 0.56 Sum of the Weighted Scores 3.62 3.62 Ineffective 1.0 Points Partially Effective 1.85 Points Effective 2.65 Points Highly Effective 3.5 Points 4.0 Points 17 Summative Rating Example (Non SGP Teacher) Example 2: Effective Teacher

Weight Weighted Score Component Raw Score Teacher Practice 3.35 0.85 2.85 Student Growth Objective 3.50 0.15 0.53 Sum of the Weighted Scores 3.38 3.38 Ineffective 1.0 Points Partially Effective 1.85 Points Effective 2.65 Points Highly Effective 3.5 Points 4.0 Points 18 Summative Rating Example (Non SGP Teacher) Example 4: Partially Effective Teacher Weight

Weighted Score Component Raw Score Teacher Practice 2.60 0.85 2.21 Student Growth Objective 2.50 0.15 0.38 Sum of the Weighted Scores 2.59 2.59 Ineffective 1.0 Points Partially Effective 1.85 Points Effective 2.65 Points Highly Effective 3.5 Points 4.0 Points 19 Component Weighting for SGP Teachers

For teachers who receive an SGP score, the scoring breakdown includes an SGO rating, an SGP rating, and a teacher practice rating (see image). The teacher practice and SGO ratings are calculated as individual components on a 1 4 scale at the district level and reported to the Department. The SGP rating is calculated by the Department and shared with the district for confidential distribution. The following pages include examples of how a summative rating will be reached. 2013 2014 Weights 45% Student Achievement 30.00 % 15.00 % Student Growth Percentile Student Growth Objectives 55.00 % 55% Teacher Practice Teacher Practice 20 Summative Rating Example (SGP Teacher)

Example 1: Highly Effective Teacher Raw Score Teacher Practice 3.60 0.55 1.98 3.90 0.30 1.17 4.00 0.15 0.60 Student Growth Percentile *77 Student Growth Objective Weight Weighted Score Component Sum of the Weighted Scores 3.75 *This is the mSGP score this particular teacher received, which converts to a 3.9 on the SGP Conversion Chart. 3.75 Ineffective 1.0 Points Partially Effective 1.85 Points Effective

2.65 3.5 Points Points Highly Effective 4.0 Points 21 Summative Rating Example (SGP Teacher) Example 3: Effective Teacher Raw Score Teacher Practice 2.60 0.55 1.43 3.00 0.30 0.90 2.75 0.15 0.41 Student Growth Percentile *48 Student Growth Objective Weight Weighted Score Component Sum of the Weighted Scores 2.74

*This mSGP score converts to a 3.0 on the SGP Conversion Chart. 2.74 Ineffective 1.0 Points Partially Effective 1.85 Points Effective 2.65 3.5 Points Points Highly Effective 4.0 Points 22 Summative Rating Example (SGP Teacher) xample 4: Partially Effective Teacher Raw Score Teacher Practice 2.50 0.55 1.38 2.40 0.30 0.72 2.75 0.15 0.41 Student Growth Percentile

*34 Student Growth Objective Weight Weighted Score Component Sum of the Weighted Scores 2.51 *This mSGP score converts to a 2.40 on the SGP Conversion Chart. 2.51 Ineffective 1.0 Points Partially Effective 1.85 Points Effective 2.65 3.5 Points Points Highly Effective 4.0 Points 23 FIND OUT MORE: www.nj.gov/education/AchieveNJ educatorevaluat[email protected] j.us 609-777-3788