SGO2.0: 2.0: SGO fromCompliance Complianceto toQuality Quality from
SGO2.0: 2.0: SGO fromCompliance Complianceto toQuality Quality from Increasing SGO Quality through Better Assessments and Target Setting Revised 9.1.14 1 Changes for SGOs in 2014-15 1) All teachers set SGOs: 20%* of summative rating Teachers without an mSGP set two SGOs 20.00 % 80.00 %
Teacher Practice Teacher Practice Student Growth Objectives Student Growth Objectives Teachers with an mSGP set one or two SGOs 20.00 % 10.00 70.00 % % Teacher Practice Student Growth Percentile Student Growth Objectives
2) SGOs approved by supervisor by October 31 2 Note for Districts Using this Presentation and Resources This presentation has been designed by the Department for use by educators in districts to help them increase SGO quality. Read the notes below each slide carefully for additional information and context for the contents of the slides. (For PDF format, download file to view notes.) Links to resources in PDF format are embedded in the presentation. Other formats are available on the AchieveNJ website SGO page. Even though the contents of this presentation represent emerging best practices in SGOs and well established rules for assessment design, districts should understand that these are guidance materials only. They should be adapted and modified to meet district-specific needs and priorities. For clarification on any of the topics covered by this presentation please visit http://www.state.nj.us/education/AchieveNJ/ or email [email protected] 3 Objectives for Today 1. Clarify what SGOs are and what they are not. 2. Develop a foundational understanding of how to develop and
choose high quality assessments. 3. Investigate appropriate ways to set targets using readily available student data. 4. Develop a series of concrete next steps that will allow you to increase the quality of SGOs in your district. 4 Part 1 Clarify what SGOs are and what they are not. 5 Requirements for Student Achievement Measures TEACHNJ Act The standards for approval of educator evaluation rubrics at a minimum shall include: a provision ensuring that performance measures used in the rubric are linked to student achievement.
A Student Growth Objective is an academic goal that teachers and evaluators set for groups of students. It shall be specific and measurable, based on available student learning data, aligned to Core Curriculum Content Standards (or other standards adopted or endorsed by the State Board), and based on growth and/or achievement. 6 The Value of SGOs For Educators SGOs provide a method by which teachers can improve their practice through high quality goal setting while clearly demonstrating their effectiveness through the learning exhibited by the students for whom they are responsible. For Evaluators SGOs provide an authentic measure of teacher effectiveness that is aligned to the learning exhibited by students through an educators daily practice of teaching.
For Students When well-designed, SGOs promote reflective and collaborative teaching practices, alignment among standards, instruction and assessment, and improve student learning. 7 What SGOs Are, and What They Are Not Misconception SGOs need to be a significant addition to the work of a teacher. #1 Reality SGOs should be a reflection of what effective teachers typically do. 8
SGOs should be a reflection of what educators typically do Three of a Teachers Circles of Concern Assessment SGO Instruction Standards SGO Quality Rubric - excerpt Number of students in combined SGOs represents all or a large majority of the teachers students. Includes start and stop dates that include a significant proportion of the school year/course length. Includes a significant proportion of standards for
which the teacher is responsible during the instructional period. SGO Quality Rating Rubric 9 General and Specific SGOs General Specific 0 . 1 4 O 1 G S 13 0 2
Captures a significant proportion of the students and key standards for a given course or subject area Most teachers will be setting this type of SGO Focuses on a particular subgroup of students, and/or specific content or skill For teachers whose general SGO already includes all of their students, or those who receive an SGP 10 2014-15 SGO Form Name School Significant proportion of
students, standards and course Grade 9 Course/Subject Physics 1 Number of Students 55/55 Interval of Instruction October-April Rationale for Student Growth Objective Name the content standards covered, state the rationale for how these standards are critical for the next level of the subject, other academic disciplines, and/or life/college/career? Name and briefly describe the format of the assessment method. Standards
NJCCCS physical science 5.2.12 C, D and E NJCCCS science practices 5.1.12 A-D Impact of Standards This SGO includes all of the NJCCCS related to physics creating a foundation important for students who will take AP and/or college-level physics and is fundamental to many careers including architecture, mechanics, engineering, medicine. The SGO also includes all of the science practice standards, standards crucial in helping student become scientific thinkers. This mindset is valuable for making decisions when a large amount of information is available and must be analyzed for value and accuracy. It is critical in most academic disciplines. Assessment Physics departments common assessment administered at the end of the 3rd marking period Written: 60 multiple choice (4 choice), 5 short response questions, Practical: Students design a simple apparatus, take measurements and collect data. 2014-15 SGO Form High quality test normally 11 administered at this time #2 What SGOs Are, and What They Are Not
Misconception SGOs are an administrator-driven compliance exercise Reality SGOs are driven by teachers, supported by administrators, and centered on student learning 12 SGOs are driven by teachers, supported by administrators, and centered on student achievement Administrator-supported Provide a supportive and collaborative environment Assess quality and provide approval and final score of SGOs Teacher-driven Identify critical standards and develop assessments Use appropriate data to set ambitious and achievable
targets Monitor performance and adjust instruction as needed Student-centered What should my students learn by when? How will I ensure they learn it? How will I know they have learned it? 13 Updated Resources for 14-15 Part 2 Develop a foundational understanding of how to develop and choose high quality assessments. 15 Turn and Talk What is the relationship between assessment quality and SGO quality? 16 SGO
Quality depends upon Assessment Quality Poorly designed assessments do not accurately measure student knowledge and learning. If SGOs are based on low-quality assessments, then the SGO process cannot yield accurate or meaningful results. If SGOs do not yield accurate or meaningful results, they will fail to promote good instruction and improve student learning. 17 Types of Assessments for SGOs Teachers may use but are not limited to:
Portfolios Performance Assessments Benchmark Assessments Finals (modified as needed) Program-based Assessments Standardized Tests, e.g. AP Whether locally-developed or commercial, multiple choice or rubric-based, assessments should follow the rules of good assessment design. 18 What Does Good Assessment Look Like? 19 Elements of Assessment Design Purpose Purpose
Valid/ Accurate Inferences Reliable/ Consistent Assessment Design Accessible Align to Standards Range of Note: The elements of assessment design have Rigor/DOK been updated for the 2015-16 school year. When Note Taking Handout
accessing information about the elements of assessment design please consult the SGO 2.1 Presentation. 20 Elements of Assessment Design Begin with the End in Mind Purpose SGO assessments are measures of how well our students have met the learning goals we have set for them 21 Elements of Assessment Design Valid/Accurate Inferences Purpose Valid/ Accurate Inferences Reliable/
Consistent Assessment Design Align to Standards Accessible Range of Rigor/DOK 22 Elements of Assessment Design Valid/ Accurate Inferences 23 Elements of Assessment Design
Valid/ Accurate Inferences Valid/Accurate Inferences Why does The assessment should measure what it sets out to measure. it matter? The assessment is aligned to standards, and rigor of the instruction and What does skills, content of the course. it look like? The assessment is accessible to all students. 24 Elements of Assessment Design Align to Standards Purpose Valid/ Accurate Inferences
Reliable/ Consistent Assessment Design Align to Standards Accessible Range of Rigor/DOK 25 Analyze This Item How valid is the inference we can make about student learning using this question? How can we make this a better assessment item? Perhaps the most famous of all the arts of the Ming Era was:
A. the elaborate puzzles of the period, which were popular even in Europe. B. blue-and-white porcelain, which Europeans collected in great quantities. C. the construction of large, elaborate palaces, the finest example of which is the Imperial City in Beijing. D. high-quality Berber rugs, which are still popular today. 6.2.12.C.1.b - Trace the movement of essential commodities (e.g., sugar, cotton) from Asia to Europe to America, and determine the impact trade on the New Worlds economy and society. Handout 26 27 Item is not aligned to standards 6.2.12.C.1.b - Trace the movement of essential essential commodities commodities (e.g., sugar, cotton) from Asia to Europe to America, and determine the impact trade on the New Worlds economy and society. of Ming
the Ming Era Perhaps the most famous of all the arts arts of the Era was: A. the elaborate puzzles of the period, which were popular even in Europe. B. blue-and-white porcelain, which Europeans collected in great quantities. C. the construction of large, elaborate palaces, the finest example of which is the Imperial City in Beijing. D. high-quality Berber rugs, which are still popular today. 28 Elements of Assessment Design Align to Standards Given limited resources, especially time, on which standards do we focus our SGOs and assessments? 29
Elements of Assessment Design Align to Standards Determine the relative importance of the standard using the following criteria 1. How much time is spent teaching the standard? 2. Does the standard have value beyond the current course in: i. the next level of the subject, ii. other academic disciplines, or iii. life/college/career? 30 Determine the relative importance of the standard being taught during the SGO period* Standard Name Rating* Rank*
CCSS.ELA-LITERACY.RL.5.4 Determine the meaning of words and phrases as they are used in a text CCSS.ELA-LITERACY.RL.5.6 Describe how a narrator's or speaker's point of view influences how events are described CCSS.ELA-LITERACY.RL.5.9 Compare and contrast stories in the same genre CCSS.ELA-LITERACY.RL.5.2 Determine a theme of a story, drama, or poem from details in the text Rationale for Rating and Rank* * Answers will vary based on many factors. 31 Determine the relative importance of the standard being taught during the SGO period* Standard Name
4th Fewer Questions/Points Rationale for Rating and Rank* * Answers will vary based on many factors. 32 2014-15 SGO Form Rationale for Student Growth Objective Name the content standards covered, state the rationale for how these standards are critical for the next level of the subject, other academic disciplines, and/or life/college/career. Name and briefly describe the format of the assessment method. Standards NJCCCS physical science 5.2.12 C, D and E NJCCCS science practices 5.1.12 A-D This SGO includes all of the NJCCCS related to physics creating a foundation important for students who will take AP and/or college-level physics and is fundamental to many careers including architecture, mechanics, engineering, medicine.
The SGO also includes all of the science practice standards, standards crucial in helping student become scientific thinkers. This mindset is valuable for making decisions when a large amount of information is available and must be analyzed for value and accuracy. It is critical in most academic disciplines. 33 Using Commercial Products for SGOs 34 Elements of Assessment Design Range of Rigor/Depth of Knowledge 35 Elements of Assessment Design Range of Rigor/DOK 36 Elements of Assessment Design
Range of Rigor/DOK Range of Rigor/Depth of Knowledge An assessment that accurately reflects the range of of the course and instruction increases the Why does rigor validity of inferences educators can make about it matter? student learning. Provides access points to students of varying ability. assessment requires a range of thinking skills as What does The it look like? proposed by Blooms taxonomy and Webbs Depth of Knowledge (DOK) that reflects the rigor of the course. 37 Elements of Assessment Design Depth of Knowledge Wheel Range of Rigor/DOK
4 minute video explaining DOK using the Gettysburg Address Handout 38 Determine the Rigor of this Item What DOK level does this item represent? What modifications could you make to the question to make it more rigorous? Examine the following political cartoon and answer the following questions. 1.What does the snake in this cartoon represent? 2.Whom is the snake attacking?
Handout 39 Determine the Rigor of this Item What DOK level does this modified item represent? Examine the following political cartoon. Use details from the cartoon to: 1. Explain the symbolism of the snake in the political cartoon. 2.Explain why the artist used children to represent free press, free speech, and honest opinion. 40 Elements of Assessment Design NOT Rigor for Rigors Sake Range of Rigor/DOK A high quality assessment has a range of rigor that:
Is representative of the rigor of instructional level and content delivered in the course, and Provides stretch at both ends of ability levels 41 Elements of Assessment Design Accessible 42 Elements of Assessment Design Accessible 43 Elements of Assessment Design Accessible Accessible Assessment Why does it matter? Promotes similar interpretations of the data.
Its fair to all students. What does it look like? Provides equal access to all students regardless of personal characteristics/background and pre-existing extra-curricular knowledge. Questions and structure do not disadvantage students from certain groups or those without particular background knowledge. Appropriate modifications for students with learning plans. Format, wording, and instructions are clear. 44 Examples Directions: Directions: Choose the one answer that best solves the problem.
Choose the one answer that best solves the problem. If one card is taken at random from a deck of playing cards, what is the probability that the card will be an ace? A) 8% B) 50% C) 25% D) 10% There are 4 aces in a deck of 52 playing cards. If one card is taken at random from the deck, what is the probability that the card will be an ace? A) 8% B) 50% C) 25% D) 10% 45
Examples Directions: Directions: Choose the one word or phrase that best completes the sentence. Choose the one word or phrase that best completes the sentence. The soldiers and their wives excitedly attended the _________. A) funeral B) celebration C) meeting D) workshop The soldiers and their spouses excitedly attended
the _________. A) funeral B) celebration C) meeting D) workshop 46 Examples 47 Check for Understanding Directions: Choose the one word that completes the sentence. Directions: Choose the one word or phrase that best completes the sentence. Quarterbacks are often sacked during games _______ they do not have a good offensive line protecting them.
Some students are often late to class _______ they visit their lockers too frequently. A) even though B) although C) in spite of D) because A) B) C) D) even though although in spite of because 48 Elements of Assessment Design Reliable/Consistent
49 Elements of Assessment Design Reliable/ Consistent Reliable le Unreliab 50 Elements of Assessment Design Reliable/ Consistent Reliable/Consistent Assessment Why does it matter?
Provides information about student learning that can be trusted. What does it look like? Assessment administration and scoring is standardized and comparable. Assessment items yield consistent results over time. 51 Several Ways to Increase Assessment Reliability Ensure any preparation for assessment is consistent across teachers and students. Develop systems so that the same assessment is administered in the same way each time. Ensure scoring is done using clear criteria; use multiple scorers, cross-scoring and/or audits to increase consistency.
52 Check for Understanding Day Monday Weight Scale (lbs) 130 Bathroom Time of Day Morning Tuesday 130 Bathroom Morning
How would you describe the reliability of this scale? How about the validity of the information you get from it? 53 Elements of Assessment Design Bringing the elements together into a coherent whole Blueprint 54 Elements of Assessment Design PRIOR TO TEST DESIGN Standard and Relative Description of Importan Standard
ce of (NJCCCS, CCSS, Standard etc.) 4.NBT.B.4 Add and subtract multidigit whole numbers 4= High 3= Mediumhigh 2= Mediumlow 1= Low 4 Blueprint DURING TEST DESIGN Type of Question (multiplechoice, constructedresponse,
Depth of Question Total Point Knowledge Number/ Value/ of Question Points Percentag 4= Extended e of Test essay, etc.) Thinking 3 = Strategic Thinking 2 = Skill/ Concept 1 = Recall MC 2 #1/5 pts
MC 3 #3/5 pts CR 3 #6 /20 pts Assessment Blueprint and Completion Guide 30 pts /10% 55 Part 3 Investigate appropriate ways to set targets using readily available student data. 56
Pre-tests - The Siren Song of Simplicity 57 #3 What SGOs Are, and What They Are Not Misconception Reality SGOs are a statistically precise measure of growth based on a pre-test/posttest model of performance. SGOs are learning targets for key concepts and skills that students can be expected to master in a course based on a rough sense of where they start. 58
Important Considerations if Using the Pre-test Post-test Model Reliability of Assessment Data Especially in Pre-test Dont worry about it this doesnt count. Lack of Value for Instructional Purposes Yep, just as I thought my kids dont know any Mandarin yet. Difficult to Set Reasonable Targets Impossible to extrapolate future learning from one data point. 59 100 90 80 70 60 50 40 30 20 10 0
? ? ? 0 0.5 1 1.5 2 2.5 3 3.5 4 Time Expected learning cannot be determined using one data point. Learning Learning Predicting Student Learning Based on a Rough Sense of Where They Begin 100 90 80 70 60 50 40 30
20 10 0 0 0.5 1 1.5 2 2.5 3 3.5 4 Time Expected learning is betterdetermined using multiple measures of starting points. 60 Predict the Final Picture 61 Predict the Final Picture 62 Predict the Final Picture 63 List the information you have used or could potentially
use to determine students starting points. 1. 2. 3. 4. 5. 64 List the information you have used or could potentially use to determine students starting points. 1. 2. 3. 4. Current grades Recent test performance Previous years scores Well-constructed and administered, high-quality pre-assessments 5. Important markers of future success
65 Sample Rubric for Important Markers of Future Success Criterion Active Participant Level 4 Always prepared Engaged in all of the learning process Level 3 Mostly prepared Engaged in most of the learning process Frequently Consistently demonstrates
demonstrates intellectual Academic intellectual curiosity curiosity Independence Consistently self Usually selfmotivated and motivated and independent independent Class Attendance Never absent Rarely absent Rubric for Important Markers of Future Success Level 2 Level 1 Sometimes prepared
Engaged in some of the learning process Rarely prepared Engaged in little or none of the learning process Sometimes demonstrates intellectual curiosity Sometimes selfmotivated and independent Rarely demonstrates intellectual curiosity Rarely or never selfmotivated, frequently depends on prompting and/or teacher assistance
Sometimes absent Frequently absent 66 Physics 1 SGO Using Multiple Measures of Starting Points to Determine Three Groups* Prior Year Final Grade Current Year Test Scores Math Average Score Participates in Class Completes Homework 1
86 98.5 Yes No 2 73 92.5 Yes Yes 3 96 95
Yes Yes 4 92 85.5 Yes No 5 67 54 No No
6 69 58 No No 7 78 72.5 Yes No 8 94
80.5 No No Student ID Prior Year Math Grade <70 70 84 85 100 Current Year Test Score Average <70 70 84 85 100 Markers of Future Success Number of Future
Success Markers 0 1 2 * May be more or fewer than three groups Preparedness Group 3 2 1 Preparedness Group Number 1 2 2 1 0 0 1
0 1 1 1 1 3 3 2 2 The teacher may assign a specific preparedness group when a majority of measures indicate a specific group using the guide at left. 8 Determine Appropriate Learning Targets Determine the level of performance on the assessment that would
indicate a sense of competence/mastery of the content and skills. Modify learning targets so they are ambitious and achievable for the preparedness level of the students . Student Growth Objective* 85% of students will meet their learning targets as shown in the table below. Preparedness Group (e.g. 1,2,3) Number of Students in Each Group Target Score on SGO Assessment 1 31 90 2 63
80 3 16 75 4 15 65 *This table has an extra row for four preparedness groups. 68 Appropriate Role of the Pre-test/Post-test Model in SGOs
Where improvement in a set of skills is being evaluated When assessments are high quality and vertically aligned When pre-tests are normally used for diagnostic purposes In combination with other measures to help group students according to preparedness level Grade 1 Reading - DRA Student High Frequency Initial Markers of Word DRA Level Future Success Recognition Preparedness Group DRA Target
1. 3 25 5 2 14-16 2. 3 35 10 1 16-18
3. 3 26 8 2 14-16 69 Sample Scoring Plan for Students with Varied Starting Proficiency* Preparedness Group More than 2 years below grade 1 to 2 years below grade Above grade level to 1 year below grade level *
Attainment Level in Meeting Student Growth Objective Student Proficiency Growth on Reading Assessment (years) Exceptional 4 Full 3 Partial 2 Insufficient 1 2.0 1.5 1.0 <1.0 1.5
1.25 1.0 <1.0 1.25 1.0 0.75 <0.75 More information than just reading level should be used when determining appropriate targets. Individualized targets could be used if students dont fit into clear categories. 70 Determine Teachers SGO Score
Use and adjust ranges of student performance to derive a score that accurately reflects teachers effectiveness while taking into account the fluid nature of teaching and learning. Scoring Plan* Preparedness Group Student Target Score on Assessment 1 2 3 4 90 80 75 65 Teacher SGO Score Based on Percent of Students Achieving Target Score
*This table has an extra row for four preparedness groups. Percentages and target scores are for illustrative purposes only . Educators should tailor these numbers to best reflect their situations. 71 Consider Tailoring SGOs and Scoring Plans for Different Situations Small Class Size Number of students per group attaining differentiated learning targets Proportion of students meeting individual goals Average proficiency score in the class by group or overall Full Attainment of Objective (3 points) At least 5/7 students in group 1 will score 85% on assessment. 75% of the 12 students in class will attain their individual learning targets. The average score of the six students in the class will be 80%.
Resource Room Exceptional Attainment of Objective (4 points) Students will achieve a score of 90% or graduate from the program. Account for students who graduate from a short-term program Scoring Plans with Finer Increments Score 4.0 3.5 3.0 2.5 2.0 1.5
1.0 % Students 95 85 80 75 70 65 <65 72 Part 4 Develop a series of concrete next steps that will allow you to increase the quality of SGOs in your district.
73 Possible Next Steps Share information from this workshop with all members of your DEAC and develop a strategy for developing higher quality assessments and SGOs throughout the district. Review the materials from this workshop and plan the time and method for delivering to staff in a PD session. Ask building leaders to create an SGO assessment inventory and check quality against the elements of assessment design and item design rules. Ask teachers to identify 3 sets of data to determine student starting points. Build in time during PLC/team time for assessment development. Use the SGO quality rating rubric to determine quality of SGOs during the approval process (deadline - October 31st, 2014). Activity Handout 74 Resources
Updated SGO guidebook and forms Expanded SGO library FAQs ScIP Workshops Information www.nj.gov/education/AchieveNJ Questions [email protected] 609-777-3788 75
Source: 1) ASPE computations of CMS Federally-facilitated Marketplace data as of 5/12/2014; 2) McKinsey Analysis as of 10/22/2014. Price spread increases significantly when there are 10+ hospitals. Price Spread of Lowest Cost Silver Plans in Regions with .
The resulting debt ratios can be very different. Costs and Benefits of Debt Benefits of Debt Tax Benefits Adds discipline to management Costs of Debt Bankruptcy Costs Agency Costs Loss of Future Flexibility Tax Benefits of Debt When you borrow...
Helping verbs are used with the present participle and past participle to form some tenses. Regular & Irregular Verbs. A regular verb forms its past and past participle by adding -d or ... "Keep the dead dead." ...
Gilgamesh is a ruthless and oppressive leader, and the gods punish his prideful behavior by killing his best friend (Enkidu). Horrified by Enkidu's death and the prospect of his own demise, Gilgamesh undertakes a quest for immortality which brings him...
Acids and Bases Have you ever wondered, "Why do foods taste sour or bitter?" It all depends on whether it is acidic or basic (alkaline)… Acids taste sour Bases taste bitter There are far more significant differences between acids and...
Ref: Helen Timperley, Linda Kaser and Judy Halbert. Setting my Performance Goal and my Inquiry Focus. Appraisal conversations: Progress and next steps. What is the data/ information now showing? Evidence here will benefit my learners as next steps . What...