Farmington Public Schools

Farmington Public Schools

The Impact of No Child Left Behind (NCLB) on School Achievement and Accountability. Glenn Maleyko Wayne State University Detroit, MI October 17, 2011, Dissertation Defense No Child Left Behind (NCLB) The No Child Left Behind (NCLB) reform may be the most significant legislation affecting public education that has been enacted by the federal government during the past 35 years (Peterson & West, 2003). The Implementation of public education legislation has historically and constitutionally been the responsibility of individual states; however NCLB educational policy at the federal level sets mandates for performance standards with required consequences if they are unmet (Hess & Finn, 2004). Statement of the Problem The purpose of this study was to evaluate the effectiveness of AYP at measuring school success in order to establish the conditions for school reform at the school level and classroom level. This included an analysis of both

quantitative data and qualitative data. Refer to Handout Packet #1 Research Questions Is there a significant correlation between the schools that meet proficiency standards on the NAEP and the schools that make AYP in the sample states? Are there significant differences among the subgroups? Is there a significant correlation between the schools that meet proficiency standards on the NAEP and schools that meet proficiency standards on the state accountability assessments (STAR, MEAP, End of Grade Test, and TAKS) in the sample states? Is there a c ombination of factors that best predicts proficiency status on the NAEP and state accountability assessments? Are the demographic (categorical) characteristics and educational resources significant predictors of the schools that make AYP and fail to make AYP in the sample states? Instruments Data Analysis Tools State AYP data in the sample states along with NAEP data from grades 4 a nd 8 in the sample Pearsons

states. Product-Moment Correlation NAEP restricted school level data in mathematics and reading at the 4th and 8th grade level. State proficiency assessment data from the sample of schools where NAEP data was collected in the sample states for the years 2005 and 2007 available through state databases and websites. Pearsons Product-Moment Correlation Logistic Regression State AYP school data and demographic data that is available Logistic through state database websites Regression and the National Center for Education Statistics (NCES). What impact is AYP having on Qualitative research techniques school improvement initiatives and through interviewing principals classroom instruction? and teachers in four schools in one of the sample states. Semi-structured interview protocol and the triangulation of interview data. Sample Population

Quantitative: All schools that took the NAEP assessment in reading and mathematics at the 4th and 8th grade level during the years 2005 and 2007 from the sample states: 1. California 2. Michigan 3. North Carolina 4. Texas Qualitative: 4 schools from the Quantitative Dataset. 4 teachers and one administrator were selected from each school for the semistructured interview protocol. National Assessment of Educational Progress (NAEP) Assessment Complicated formula (Plausible Values) This data was difficult to use and it is where I spent the majority of my time with data analysis in this study A formula was created to develop school level proficiency on the NAEP which is the first of its kind Refer to Handout Packet #2 for variable definitions. Pearson Correlation Analysis r value defined Terminology Minor Moderate Large, Strong, or Sizable

Extremely large or Strong Approx. Range (+ or -) .000 to .099 (+ or -) .100 to .399 (+ or -) .400 to .549 (+ or -) .550 to 1.0 Research Question One Findings California, Michigan, some of the NC datasets, and the 8th Grade Texas dataset -- --Provided a stronger relationship with AYP and the basic level NAEP scale vs. AYP and the proficient level NAEP scale Please refer to Handout Packet #3 and #4 for NAEP basic and proficient level definitions. Michigan 8th Grade 2007 AYP NAEP BAS IC NAEP PROF TOTAL STATEAYP level level REVENUE EDPER

NATA MPER 1 .429 ** .055 ** -.155 ** -.396 ** .064 ** .429 ** 1 .079 ** -.261 ** -.666 ** .074 ** .055 ** .079 ** 1 .093 ** -.202 ** -.045** ASIANPER

BLACKPER ELLPER SPECIALEDPER STATEAYP .139 ** -.627 ** .027 ** .522 ** -.023 ** -.057** NAEP BAS IC level .126 ** -.717 ** -.256 ** .665 ** -.343 ** -.256** .053 ** -.068 **

-.055 ** .082 ** -.048 ** -.058** STATEAYP NAEP BA SIC level NAEP PROF level NAEP PROF level HISPAN ICPER WHITEPER Please refer to Handout Packet #5 North Carolina and Texas Most of the North Carolina datasets produced a stronger relationship with AYP and the proficient level NAEP scale vs. the basic Level NAEP scale The Texas datasets had a very minor relationship with AYP and the NAEP proficiency at the proficient and basic levels North Carolina 2007 4th Grade Dataset NAEP BAS IC

NAEP PROF TOTAL STATEAYP level level REVENUE EDPER NATA MPER STATEAYP 1 .336 ** .418 ** -.048 ** -.445 ** -.124 ** NAEP BAS IC level .336 ** 1 .214 ** .141 **

-.548 ** -.394 ** .418 ** .214 ** 1 .079 ** -.581 ** -.106 ** ASIANPER BLACKPER ELLPER SPECIALEDPER .021 ** -.426 ** -.215 ** .475 ** -.255 ** -.204 ** .103 **

-.229 ** -.238 ** .475 ** -.242 ** -.087 ** .264 ** -.375 ** -.273 ** .432 ** -.199 ** -.103 ** NAEP PROF level STATEAYP NAEP BAS IC level NAEP PROF level HISPAN ICPER WHITEPER Please refer to Handout Packet #6 Research Question 2 Results: Michigan, California, and some of the NC datasets: Produced Pearson Correlation results and

Logistic Regression results showing a stronger relationship with the basic level NAEP scale and state accountability assessments vs. the proficient level NAEP scale and state accountability assessments Research Question 2 Results California had the greatest increase from the constant model vs. the predicted model showing that the logistic model had a strong influence Thus the logistic model was useful in California. It is important to emphasize that the logistic regression is a more powerful tool of analysis for predicting the probability that a school will meet proficiency status. Constant Model 2005 California 8th Grade Observed Step 1 Predicted STATEMATHprofstatus did not meet met

Proficiency Proficiency .00 1.00 did not meet proficiency 0 148293 .0 186618 100.0 status met proficiency status 0 Overall Percentage a. The cut value is .500 Please refer to Handout Packet #7 55.7 Predicted Model 2005 California 8th Grade Observed Predicted did not meet met Step 1

STATEMATHprofstatus Proficiency Proficiency .00 1.00 did not meet proficiency 115866 32427 78.1 151409 81.1 status met proficiency status 35209 Overall Percentage a. The cut value is .500 Please refer to Handout Packet #7 79.8 Research Question 2 Results Texas and California had similar sample sizes and population demographics including a higher level of the HISPANICPER variable

However, the Texas results were not very useful and there was a very minor relationship between the state accountability assessments and the NAEP as measured by the Logistic Regression and the Pearson Correlation results Research Question 2 Results Explanation: An extremely high percentage of schools in Texas met proficiency status resulting in low variability with the dependent variable. Texas had the weakest relationship with NAEP at both the basic and proficient level among the 4 sample states This suggests less rigor with the Texas state accountability assessments Texas 2007 4th Grade Mathematics: Pearson Results NAEP BAS IC NAEP PROF TOTAL STATE Math level Math

level Math REVENUE EDPER NATA MPER STATE Math 1 -.008** .046** -.069** -.109** .052** NAEP BAS IC Math -.008** 1 .061** .001 -.001 .069** .046** .061**

1 .005* -.608** .220** ASIANPER BLACKPER ELLPER SPECIALEDPER .047** -.048** -.035** .062** -.015** .043** .054** -.272** .079** .090** .014**

.026** .506** -.214** -.545** .654** -.441** -.037** NAEP PROF Math STATE Math NAEP BAS IC level NAEP PROF level HISPAN ICPER WHITEPER Please refer to Handout Packet #8 Texas 2007 4th Grade Math Logistic: Constant Model Observed Predicted did not meet met Step 1 STATEMATHprofstatus Proficiency Proficiency

.00 1.00 did not meet proficiency 0 979 .0 161201 100.0 status met proficiency status 0 Overall Percentage a. The cut value is .500 Please refer to Handout Packet # 9 99.4 Research Question 2 Findings The California and NC logistic regression results were the most useful in this study The Pearson results in Michigan were useful showing that that was a strong association with the NAEP scale basic level and the state accountability

assessment results Texas Results were the least useful Michigan 2005 8th Grade Mathematics NAEP BAS IC NAEP PROF TOTAL State Math level Math level Math REVENUE EDPER NATA MPER STATE Math 1 .640** .150** -.280** -.666** .098** NAEP BAS IC Math

.640** 1 .194** -.267** -.705** .085** .150** .194** 1 .166** -.355** -.123** ASIANPER BLACKPER ELLPER SPECIALEDPER .187** -.704** -.020**

.682** -.137** .128** .167** -.760** -.091** .749** -.098** .074** .401** -.177** -.082** .155** -.051** -.082** NAEP PROF Math STATE Math NAEP BAS IC level NAEP PROF level HISPAN ICPER WHITEPER Please refer to Handout Packet #10

Research Question 2 Results: NC North Carolina produced some of the most intriguing results DUMMYREV had a positive impact in NC 2007 4th and 8th grade mathematics results produced the largest r value in relationship to the proficient level NAEP scale The Logistic Regression results also showed that there was a strong relationship with the 2007 NC state accountability assessments at the proficient level and the NAEP assessments in mathematics. 2007 NC 4th Grade Math NAEP BAS IC NAEP PROF TOTAL STATE Math level Math level Math REVENUE EDPER NATA MPER STATE Math 1

.079** .596** .186** -.635** -.098** NAEP BAS IC Math .079** 1 .091** .016** -.248** -.124** .596** .091** 1 .127** -.679** -.179** ASIANPER BLACKPER

ELLPER SPECIALEDPER .156** -.485** -.300** .536** -.190** -.163** .092** -.193** .058** .177** .044** -.018** .190** -.441** -.350** .563** -.259**

-.112 ** NAEP PROF Math STATE Math NAEP BAS IC level NAEP PROF level HISPAN ICPER WHITEPER Please refer to Handout Packet #11 2007 North Carolina Math 95% C.I.for EXP(B) B S.E. Wald df Sig. Exp(B) Lower Upper Step 1 DUMMYREV 1.314 .024 2901.049 1

.000 3.721 3.547 3.903 EDPER -9.461 .120 6220.284 1 .000 .000 .000 .000 NATAMPER 15.782 .858 338.117 1 .000

7148837.835 1329370.4 3.844E7 90 ASIANPER 36.833 1.190 957.663 1 .000 9.913E15 9.618E14 1.022E17 BLACKPER 4.540 .937 23.483 1 .000 93.682 14.935

587.631 HISPANICPER 10.850 .921 138.735 1 .000 51541.625 8473.310 313518.44 9 WHITEPER ELLPER 13.419 6.355 .866 240.022 1 .000 672812.255

123200.24 3674313.4 9 48 .216 862.377 1 .000 575.175 376.362 879.012 SPECIALEDPER -5.825 .188 957.691 1 .000 .003 .002 .004 NAEPMATHPRO 12.064 871.625

.000 1 .989 173541.145 .000 . .028 1626.099 1 .000 3.138 2.969 3.318 871.625 .001 1 .978 .000 Fbasic NAEPMATHPRO 1.144 Fprof

Constant -24.457 Please refer to Handout Packet #12 Research Question 2 However, NC changed the reading cut scores in 2007, thus there was a very weak relationship with the reading assessments in 2007 Cut score manipulation was one of the findings in the literature review (Guilfoyle, 2006; Harris, 2007; Sunderman, et al. 2005) Other cut score manipulations, Michigan 2011 example, College Readiness Research Logistic results were research NC and California the most useful Question 3 Regression similar to question 2 results Logistic Results

were The Texas logistic regression results were less useful supporting the finding that there was a low level of rigor with the Texas assessments as a high percentage of schools met AYP in Texas This was an interesting finding in itself as almost all schools in Texas met AYP during the 2005 and 2007 school Economically Disadvantaged (EDPER) variable: All 3 Research The EDPER variable questions produced a consistent negative association with AYP proficiency. The results were generally found to be moderate to sizable This was the most consistent finding in the study with both the Pearson correlation results and the logistic regression. California 2007 8th Grade Mathematics NAEP BAS IC NAEP PROF TOTAL STATE Math

level Math level Math REVENUE EDPER NATA MPER STATE Math 1 .567 ** .242 ** -.299 ** -.516 ** .029 ** NAEP BAS IC Math .567** 1 .270 ** -.300 ** -.688 ** .100 **

.242** .270 ** 1 -.192 ** -.516 ** -.004 * ASIANPER BLACKPER ELLPER SPECIALEDPER .301** -.193 ** -.463 ** .465 ** -.275 ** -.203 ** .408** -.131 ** -.665 ** .626 **

-.543 ** -.056 ** .279** -.145 ** -.443 ** .440 ** -.278 ** -.129 ** NAEP PROF Math STATEAYP NAEP BAS IC level NAEP PROF level HISPAN ICPER WHITEPER Please refer to Handout Packet #13 NC 2005 8th Grade Mathematics 95% C.I.for EXP(B) B S.E. Wald df Sig.

Exp(B) Lower Upper .797 .031 676.152 1 .000 2.220 2.090 2.357 EDPER -6.638 .147 2042.765 1 .000 .001 .001 .002

NATAMPER -15.537 .387 1607.696 1 .000 .000 .000 .000 ASIANPER 44.504 .824 2920.013 1 .000 2.127E19 4.235E18 1.069E20 BLACKPER -11.101

.198 3141.020 1 .000 .000 .000 .000 HISPANICPER -6.538 .331 390.921 1 .000 .001 .001 .003 DUMMYWHITE -.903 .041 475.658

1 .000 .406 .374 .440 ELLPER -10.689 .273 1534.364 1 .000 .000 .000 .000 SPECIALEDPER -2.480 .138 324.600 1 .000

.084 .064 .110 515.240 .001 1 .970 2.390E8 .000 . 354.704 .002 1 .964 8534996.5 .000 . Step 1 DUMMYREV NAEPMATHPROFb 19.292 asic NAEPMATHPROFp 15.960 rof Constant

61 -6.200 515.240 .000 1 .990 .002 Please refer to Handout Packet #14 Demographic Variables 3 Research Questions SPECIALEDPER and ELLPER variables produced inconsistent results: ELL in California a greater negative impact NATAMPER, BLACKPER, and HISPANICPER variables were not consistent among the different datasets. However, BLACKPER in Michigan produced a consistent negative influence on the probability that a school met proficiency status Demographic Variables 3 Research Questions TOTALREVENUE produced minor to moderate positive associations with State AYP and the accountability assessments TOTALREVNUE in NC had a strong

positive association with the NAEP proficient level scale in comparison to State AYP Demographic Characteristic Findings from 3 Research Questions The ASIANPER and WHITEPER were inconsistent but generally had a positive influence on state accountability results Research Question 4: School Population: Each School had different demographics Blue School, low ED population Red school high ED population Orange school high ED and ELL population Purple School high ED located in an urban setting. Experience with restructuring. Please refer to Handout Packet #15 Research Question 4 Findings AYP identification and sanctions will not lead to improvement based on participant responses Sanction provisions lacking scientific

evidence of effectiveness. Literature review aligned with participant responses Ex. Red School principal Dropping down the hammer will not work Additional support is needed. Research Question 4 Findings Purple School went through the NCLB restructuring process. Did not feel restructuring led to improvement. The removal of one grade provided change and actually gave the school a better statistical chance of making AYP as identified in the literature review. Research Question 4 Findings EDPER variable and Triangulation: The literature review, quantitative dataset and the qualitative data established that this variable had the greatest impact on student achievement as measured by AYP and the standardized assessments. Blue School principal: One of the only respondents that felt the ED population did not make a difference with AYP Blue School had less than 3% ED.

Research Question 4 Findings: Social Capital Social Capital Influence Researchers (Elmore, 2002; Harris, 2007; Mathis, 2004: Mathis, 2006) contend that AYP is measuring school demographics and the social capital that students bring to the school instead of school effectiveness. If this is the case, then the external validity of the AYP measurement formula is questionable. Orange School Felt that ELL students had the greatest impact but their ELLPER population matched the EDPER population Most Research Question 4 Findings: School Funding participants felt that school funding was key. Title One funds and additional resources were important when servicing ED students. However, interesting that Blue School did not feel this way. Funding in their district was lower than others. However, they had a greater level of social capital in the school. Probable that Social Capital with an Effective SIP was key to their success. Research Question 4: School Funding a two

tiered variable TOTALREVNUE had a positive impact or a mild negative impact with the quantitative dataset The qualitative analysis helped to create a finding listing TOTALREVENUE as a two tiered variable Funding is important but step two is how the funds are used: Effective or not effective Research Question 4 Findings: Positive Consequence Focus on SIP plan and increased focus on scientific strategies and research based best practices (ex. Differentiated instruction) Increased Collaboration with faculty Increased focus on analyzing data and electronic systems Increased focus on subgroup populations and individual student datasets at the classroom level Research Question 4: Philosophical Intent vs. Negative Impact or Unintended Consequences Increased pressure on administration:

Exception Blue school Additional paperwork and tasks for administration Stress on teachers and administrators Possibility that some educators would not want to work in schools labeled as failing. Merit pay, etc. Red School principal response Well, I think its narrowing the curriculum. And its. Taking away some of the choices kids have because they have to, you know, have to do all math. It kind of bothers me because I, you know, I think the purpose of education is to also help a child become more well rounded. But they also need to know math. Research Question 4: Negative Impact or Unintended Consequences Low Morale. Ex. Orange school Implemented Effective SIP based on my triangulation of data but not getting the AYP results. One subgroup had a negative impact Less focus on higher level thinking skills needed for success

after leaving the public school system Research Question 4 : Internal Capacity No support at the school level for increased internal capacity and development Many researchers in the field posit that the AYP formula is faulty because it relies on sanctions and punishments and fails to provide schools with the internal capacity to make change (Abdelmann & Elmore, 1999; Elmore, 2002; Fullan, 2006; Schoen & Fusarelli, 2008). Practical Implications Data analysis is a positive outcome of AYP. There was an increased focus on subgroups. More support is needed at the building level to support schools with implementing effective SIP plans Potential to make a great difference via accountability reform. However: We need to create an effective measurement system to measure school progress This study shows that the current

AYP measurement system is not effective at measuring school success and improvement. Researchers maintain there is a lack of consistency with AYP that results in reliability issues with using AYP in order to measure school effectiveness (Elmore, 2002; Guilfoyle, 2006; Kane and Douglas, 2002; Linn & Haug, 2002; Porter et. al, 2005; Scheon & Fusarelli, 2008; Wiley et al. 2005). Theoretical Implications Single Measure Standardized Accountability systems result in unintended consequences If a single annual test were the only device a teacher used to gauge student performance, it would indeed be inadequate. Effective teachers assess their students in various ways during the school year. (Guilfoyle, 2006, p. 13) Theoretical Implications The current system has a heavy reliance on standardized assessments (one size fits all measure) A multiple Measures approach to evaluating school effectiveness would be more effective Growth Data formulas Use of the NAEP

National Curriculum and use of NAEP A move towards a National Curriculum and the Common Core Standards is gaining momentum As mentioned, This was the first study that could be found which came up with a school level proficiency measure using the NAEP Summary and Questions

Recently Viewed Presentations

  • Impact of Parameter Variations on Multi-core chips E.

    Impact of Parameter Variations on Multi-core chips E.

    Impact of Parameter Variations on Multi-core chips E. Humenay, D. Tarjan, K. Skadron Department of Computer Science University of Virginia Motivation Process variations are projected to severely impact the yield of high-performance semiconductors Multi-core architectures have become the future trend...
  • kovariksph.weebly.com

    kovariksph.weebly.com

    Porocytes. Poriferans do not have any muscle cells, so their movement is rather limited. However, some poriferan cells can contract in a similar fashion as muscle cells. Porocytes which surround canal openings and pores can contract to regulate flow through...
  • Runtime Techniques for Efficient and Reliable Program Execution

    Runtime Techniques for Efficient and Reliable Program Execution

    Uses a backward pass to transitively recover object death points. Hundreds of times slowdown even for small programs (e.g., 752X for DaCapo-small) GC-based approximation. The collection of an object is treated as its death. Imprecise for many applications (e.g., all...
  • Csr - دانشگاه علوم پزشکی اصفهان

    Csr - دانشگاه علوم پزشکی اصفهان

    persistent CSR. Vision usually improved to 95% of its original level by 3 months without specific treatment. Recurrences occur in 30% of patients and in a very small number, the condition may become chronic. In these patients the treatment options...
  • Attendee Introduction

    Attendee Introduction

    gomez r daniel. pvc plastic pipe cement. leyba jose david. pvc-abs-cpvc. satin wall and trim. plasite 7122 part a. calcium carbonate marble chunks. rust stop. fluorinert electronic liquid perfluoro cmpds. standard, berrylium. beryllium spectrometric standard solution. hi gloss canary yellow....
  • A Short History of SDN

    A Short History of SDN

    Pantou/OpenWRT. Ofsoftswitch13. Indigo. Controller compliant with OpenFlow std. POX. NOX. MUL. Maestro. Available Commodity Switches compliant with OpenFlow std. Hewlett-Packard 8200zl, 6600, 6200zl, Brocade 5400zl, and 3500/3500yl. IBM NetIron CES 2000 Series
  • Estimation I

    Estimation I

    Arial Palatino Linotype Century Gothic Courier New Calibri Symbol Symbol MT Executive 1_Executive Microsoft Equation 3.0 Point Estimation Overview Big Picture Point Estimator Maximum Likelihood Estimation Examples/Exercises Method of Moments Examples/Exercises Sufficient Statistics Sufficient Statistics Examples/Exercises Exponential Family ...
  • Template for designing a research poster

    Template for designing a research poster

    biological synapse, and . B) memristive synapse. The horizontal coordinate is the relative timing of spikes, ΔT, between pre and post-synaptic neurons, and the vertical coordinate is the change in strength of the synapse, ξ(ΔT). [5] Hebbian theory of learning:...