Educational Evaluation and Policy Analysis

Includes bibliographical references.

The effects of student coaching: an evaluation of randomized experiment in student advising -- Classmates with disabilities and students' noncognitive outcomes -- Assessing the use of aggregate data in the evaluation of school-based interventions: implications for evaluations research and state policy regarding public-use data -- Teacher mobility and financial incentives: a descriptive analysis of denver's procomp -- Collective bargaining, transfer rights, and disadvantage schools -- Keeping sight of the forest through the trees: response to "collective bargaining, transfer rights, and disadvantaged schools" -- Focusing on fundamentals: a reply to koski and horng.

[Article Title : The Effects of Student Coaching: An Evaluation of a Randomized Experiment in Student Advising / Eric P. Bettinger and Rachel B. Baker, p. 3-19] Abstract: College graduation rates often lag behind college attendance rates. One theory as to why students do not complete college is that they lack key information about how to be successful or fail to act on the information that they have. We present evidence from a randomized experiment which tests the effectiveness of individualized student coaching. Over the course of two separate school years, InsideTrack, a student coaching service, provided coaching to students attending public, private, and proprietary universities. Most of the participating students were nontraditional college students enrolled in degree programs. The participating universities and InsideTrack randomly assigned students to be coached. The coach contacted students regularly to develop a clear vision of their goals, to guide them in connecting their daily activities to their long-term goals, and to support them in building skills, including time management, self-advocacy, and study skills. Students who were randomly assigned to a coach were more likely to persist during the treatment period and were more likely to be attending the university 1 year after the coaching had ended. Coaching also proved a more cost-effective method of achieving retention and completion gains when compared with previously studied interventions such as increased financial aid.;[Article Title : Classmates With Disabilities and Students' Noncognitive Outcomes / Michael A. Gottfried, p.20-43] Abstract: The increasing trend of placing students with disabilities in general education classrooms has raised questions among researchers, policymakers, practitioners, and parents about classmate peer effects on all students. However, little is known about the peer effects of classmates with disabilities on the outcomes of other students in the classroom; no research has evaluated these peer effects on other students' noncognitive outcomes though they are highly predictive of schooling and lifelong success. The purpose of this study is to fill this research gap by using quasi-experimental methods on a nationally representative data set (i.e., Early Childhood Longitudinal Study-Kindergarten Class) of elementary school students to examine the peer effects of classmates with disabilities on five noncognitive scales for classmates without disabilities. The findings indicate that students with a greater number of classmates with disabilities have higher externalizing and internalizing behavioral problems and lower frequencies of self-control, approaches to learning, and interpersonal skills. The findings are differentiated by disability category of a student's classmates and are moderated by individual and contextual factors. Implications for policy and practice are addressed.;[Article Title : Assessing the Use of Aggregate Data in the Evaluation of School-Based Interventions: Implications for Evaluation Research and State Policy Regarding Public-Use Data / Robin T. Jacob, Roger D. Goddard, and Eun Sook Kim, p. 44-66] Abstract :It is often difficult and costly to obtain individual-level student achievement data, yet, researchers are frequently reluctant to use school-level achievement data that are widely available from state websites. We argue that public-use aggregate school-level achievement data are, in fact, sufficient to address a wide range of evaluation questions and the use of this data is more appropriate than commonly thought. Specifically, we explore (a) when point estimates and standard errors differ between models that use individual student-level data and those that use aggregate school-level data, (b) the potential for conducting subgroup and nonexperimental analyses with aggregate data, and (c) the metrics that are currently available in state public-use data sets and the implications these have for analyses.;[Article Title : Teacher Mobility and Financial Incentives: A Descriptive Analysis of Denver's ProComp / Eleanor S. Fulbeck, p. 67-82] Abstract :Extensive teacher mobility can undermine policy efforts to develop a high-quality workforce. In response, policymakers have increasingly championed financial incentives to retain teachers. In 2006, the Denver Public Schools adopted an alternative teacher compensation reform, the Professional Compensation System for Teachers ("ProComp"). Using longitudinal teacher-level data from 2001-2002 to 2010-2011, I estimate hazard models that identify the relationship between ProComp and teacher mobility. Specifically, I compare mobility patterns of teachers who received a ProComp incentive with those who did not, with special attention to teacher mobility in high-poverty schools. Results suggest receiving a ProComp incentive is associated with a significant decrease in the odds of departure. This appears to be driven by a decrease in a teacher's odds of leaving the district rather than moving to a new school within the district, by voluntary ProComp participants and by teachers who receive incentives that total more than $5,000.;[Article Title : Collective Bargaining, Transfer Rights, and Disadvantaged Schools / Sarah F. Anzia and Terry M. Moe, p. 83-111] Abstract : Collective bargaining is common in American public education, but its consequences are poorly understood. We focus here on key contractual provisions-seniority-based transfer rights-that affect teacher assignments, and we show that these transfer rights operate to burden disadvantaged schools with higher percentages of inexperienced teachers. We also show that this impact is conditional: It is substantial in large districts, where decisions are likely to follow rules, but it is virtually zero in small districts, where decisions tend to be less formal and undesirable outcomes can more easily be avoided. The negative consequences are thus concentrated on precisely those districts and schools-large districts, high-minority schools-that have been the nation's worst performers and the most difficult to improve.;[Article Title : Keeping Sight of the Forest Through the Trees: Response to "Collective Bargaining, Transfer Rights, and Disadvantaged Schools" / William S. Koski and Elieen L. Horng, p. 112-119] Abstract : In this invited response to Moe and Anzia (2014), we describe both the points of convergence and divergence between our prior research (2007a, 2007b) and that of Moe (2005) and Moe and Anzia (2014). We also respond to Moe and Anzia's critique of our published work. Moe and Anzia's study helps to refine the policy discussion around seniority preferences in teacher collective bargaining agreements by providing further evidence that such preferences may exacerbate the teacher quality gap in particular settings - specifically, large, likely highly bureaucratic, elementary school districts - a finding that is consistent with a finding from our earlier research. However, we believe that those significant findings are limited to certain school districts, and we were unable in our prior research to conclude that the strength of seniority preferences consistently and systematically exacerbates the teacher quality gap within and among all school districts. This is the point of divergence between our work and that of Moe and Anzia. Consequently, we caution that merely banning seniority preferences may not have widespread, long-term effects on closing the teacher quality gap.;[Article Title : Focusing on Fundamentals: A Reply to Koski and Horng / Sarah F. Anzia and Terry M. Moe, p. 120-123] Abstract : Koski and Horng argue that our study of seniority-based transfer rules is narrow andthat is findings only apply under limited circumtances.