The Sustained Impact of Future Forward
Future Forward is an early primary program that leverages both one-on-one tutoring and parent engagement to develop the literacy skills of struggling readers. In 2010, Future Forward was awarded an i3 grant, under the name of SPARK, to develop and test its impact in seven Milwaukee Public Schools primarily serving low-income Black and Latinx students. The results of a randomized control trial evaluation found Future Forward had positive and statistically significant impacts on foundational literacy, reading achievement, and school attendance (Jones & Christian, 2020). In the current study, we follow study participants five years past the end of the study, from 2016 to 2020, to determine if the benefits gained were sustained.
The results show that former Future Forward participants continued to score higher on the school district’s benchmark Star Reading assessment after the i3-funded study and program ended. The sustained impact of Future Forward was equivalent to approximately 1/3 year of reading development according to national norms, and 1/2 year according to local norms. We also found that former Future Forward students were less likely to be placed into special education. Considering the added cost associated with providing specialized services to ten additional students, Future Forward could ultimately save MPS over one million dollars.
Although the results of this study are positive, it is important to put the impact of Future Forward in the context of a school system that is failing students of color. Although Future Forward had a large sustained impact on reading achievement, former Future Forward participants were still reading several years below grade level by middle school. So while the impact of Future Forward is real, it must be a part of larger efforts in educational resource redistribution to earnestly provide what is owed to students and families (Ladson-Billings, 2006).
Impact evaluation of the 2020-21 Future Forward Program
In 2017, Education Analytics (EA) received an Education Innovation and Research Mid-Phase grant from the U.S. Department of Education to expand FF to 14 schools in seven school districts across three states. The randomized control trial study of the 2019-20 FF program was designed to examine the effectiveness of its scale-up to 14 schools and its impact on reading achievement, social emotional learning (SEL), and school attendance. Regarding scale-up, nearly all participating students received one-on-one tutoring as intended. Fewer students received the full implementation of family engagement (FE) however. In spring of 2020, schools were shut down nationwide because of the COVID-19 pandemic. The disruption on schools limited our study to testing the impact of FF on school attendance only. FF was found to have a statistically significant, positive impact on school attendance. The school attendance of FF participants was 1.3 percentage points higher than students who received Business-as-Usual (BAU) literacy instruction. Further, its impact was greater with Black students, male students, and students with a history of low school attendance. Students at the intersection of these groups (Black male students with low school attendance) were particularly affected, improving their school attendance by six percentage points and 4.2 fewer days absent from
The Results from a Multi-Site Regression Discontinuity Impact Study of the Future Forward Literacy Program
Future Forward (FF) is an early elementary literacy program that pairs one-on-one tutoring, provided by volunteers, with parent engagement. Previously, FF, under the name of SPARK, was awarded an Investing in Innovations (i3) grant to test its impact in seven Milwaukee schools. Two randomized studies of FF/SPARK conducted as part of the i3 grant found FF/SPARK to have positive impacts on literacy, reading achievement, and school attendance. In 2017, Education Analytics received an Education Innovation and Research (EIR) Mid-Phase grant to establish FF in 14 additional schools across three states. During the 2018-19 school year, 12 schools participated in a multi-site regression discontinuity (RD) study of its impact on the literacy development of 222 students. While we did not find a statistically significant impact of FF, the magnitude of the impact estimate was similar to what was reported in the i3 studies. Limitations inherent to RD studies, and implementation challenges, hindered our ability to reliably measure the impact of FF.
Reorganizing the Professional Culture in Wisconsin Schools to Eliminate Racial Achievement Gaps
In this longitudinal study of Wisconsin schools, we examined the impact that schools with a strong professional culture have on the English Language Arts (ELA) and Math achievement of Black, White, and Latinx students. The results suggest much of the large achievement gap between Black, White, and Latinx students is attributable to inequities in the quality of schooling students receive. There are stark differences between the strength of the professional culture in schools serving Black, White, and Latinx students. These differences explain a great deal of why the achievement of Black and Latinx students lags so far behind that of White students. To put it simply, White students attend better schools. Given this, it is clear that the education system in Wisconsin represents a racist institution. However, this study demonstrates the impact that improving school professional culture has on achievement gaps. If more Wisconsin schools serving Black and Latinx students were organized to improve, with a strong professional culture, our school system would be more equitable, more anti-racist, and achievement gaps would be greatly reduced.
Wisconsin families share their experiences with home/remote learning during COVID-19
In March of 2020, the COVID-19 pandemic forced the closing of Wisconsin schools and education to move online. In response to this, as part of the Wisconsin Educator Effectiveness Research Partnership (WEERP), the Office of Socially Responsible Evaluation in Education (SREED) at the University of Wisconsin in Milwaukee worked with the Wisconsin-Minnesota Comprehensive Center for Region 10 (WMCC) and the Wisconsin Department of Public Instruction (DPI) to develop anonymous Family, Teacher, and Student Home/Remote Learning surveys. Districts can use these surveys to inform the development and monitoring of their Home/Remote Learning system. Both the Family and Student Home/Remote Learning surveys are available in English, Spanish, Chinese, and Hmong. The surveys are available to districts free of charge through links on our website at https://uwm.edu/sreed/distance-learning/. As of July 2020, 16 Wisconsin school districts had signed up to use the Family Home/Remote Learning Survey and 3,227 families had completed a survey.
What predicts the college success of former MPS students?
As part of our evaluation of the MPS Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) grant, we analyze the connection between measures of high school performance, such as AP course participation, participation in GEAR UP programming, ACT scores, and GPA with first-year college GPA. Data were provided by the Milwaukee Public Schools (MPS), the University of Wisconsin in Milwaukee (UWM), and Milwaukee Area Technical College (MATC) for students who graduated GEAR UP schools between 2015 and 2018 and then attended either UWM or MATC.
- Although we did not find a connection between GEAR UP participation and college GPA, nearly all students who graduated high school and attended MATC or UWM had participated in several GEAR UP activities. This complicated our ability to measure impact and suggests GEAR UP programming was valued by nearly all students who ultimately went to college.
- Measures of high school GPA were the only unique predictors of of college GPA.
- Although students who participated in AP classes tended to have higher college GPAs, this relationship disappeared after accounting for high school GPA.
- Although students with higher ACT scores tended to have higher college GPAs, this relationship also disappeared after accounting for high school GPA.
View full report here GU Post Secondary Prediction Study
Focus Group of former GEAR UP Students Supported by the Educational Opportunity Program at Marquette University
In June 2019, eight Marquette University undergraduate students of color were convened for a focus group at the Educational Opportunity Program (EOP) in Coughlin Hall on the Marquette University campus. Students were asked to discuss their experiences in both programs and to make suggestions for how access to college and success in college could be supported more effectively for other MPS students like them.
MPS GEAR UP Final Report of Participation and High School Graduation
In this report, we summarize student participation in Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) grant across all seven years of the initiative in the Milwaukee Public Schools. 6,888 students participated in at least one GEAR UP activity. Over 90% of participants were either African American or Latinx and were from low-income families.
The improvement in graduation rates realized by specific GEAR UP schools was associated with how well they implemented the initiative. The two schools that demonstrated the greatest improvement in graduation rates, also demonstrated more intensive GEAR UP participation (students participated across more years and in more activities). This suggests that the impact of GEAR UP on high school graduation is locally determined by the effectiveness of a school’s implementation of the initiative.
Education in the time of COVID-19: Lessons from a school in Hong Kong
This report presents information gathered from one American international school in Hong Kong that began their transition to distance learning in February. As the COVID-19 pandemic continued to stress community and school resources, this school engaged in an internal continuous improvement and monitoring process. While its data collection activities were intended to inform internal decisions, the lessons learned from these efforts may be of use to other schools struggling through their own transition to distance learning while simultaneously dealing with the impact of a pandemic in their community. The hope is that this school’s experiences will provide educators in the United States guidance for how to accomplish and sustain this extremely difficult task.
- The school designed an effective distance learning process, using a variety of tools to facilitate learning.
- As the school remained closed, over time students began to feel more socially isolated.
- Teachers experienced a great deal of stress, both related to the workload required to implement distance learning and concerns about the safety of their families.
- Ongoing and consistent communication throughout the transition has proven critical for its success and the well-being of employees. Parents, students, and employees have also appreciated opportunities to share their experiences with administration.
- Teachers with children face a daunting task of both continuing their work as a teacher AND leading the home schooling of their children.
- Most teachers felt their instructional skills have improved as a result of the switch to distance learning.
Research Brief: The Utility and Cost of Requiring Wisconsin Teachers Pass the FoRT and edTPA assessments
In this study, conducted by The Office of Socially Responsible Evaluation in Education at the University of Wisconsin Milwaukee and the University of Wisconsin System, we explore the utility and impact of the requirement that preservice teachers meet a minimum score on the both the Educative Teacher Performance Assessment (edTPA) and the Wisconsin Foundations of Reading Test (FoRT).
WEERP Brief Nov 2019 – Race, Relational Trust, and Teacher Retention
In this report we explore relational trust as a possible explanation for: 1) why most Wisconsin schools do not have have any teachers of color and 2) the schools with the most students of color have the most difficulty retaining teachers.
We found African American teachers have lower trust with other teachers and are considerably more likely to transfer or leave public education than White teachers. This contributes to the shortage of African American teachers in schools across the state.
We also found White teachers reported less trust with African American or Latinx principals and between teachers, when fewer teachers were White. White teachers were more likely to transfer out of these types of schools, which generally have more students of color. This helps explain the teacher retention challenges experienced in schools comprised almost entirely of students of color.
The Impact of the Wisconsin Educator Effectiveness Process on New Teacher Turnover
In this report, we found the effective implementation of the teacher performance feedback process promotes the retention of new teachers. Providing new teachers accurate performance feedback led to improved trust with their principal, more school commitment and greater retention.
WEERP – Wisconsin Urban Leadership Institute Brief – September 2019
The five largest school districts began working with the Urban League of Greater Madison and the Wisconsin Department of Public Instruction three years ago to develop strategies for promoting equitable education opportunities for students of color. Their efforts resulted in the Wisconsin Urban Leadership Institute. This report presents the results from the first year of this important initiative.
The Impact of the Wisconsin Educator Effectiveness Process on Student Achievement
In this report, we found schools that improved aspects of the teacher performance feedback process also demonstrated improved student achievement results. Schools that increased the opportunities teachers have to use feedback and the amount that teachers used feedback improved their reading and math achievement results.
The Associations between How Schools use the Results of Teacher Evaluations, Principal Preparedness for Completing Teacher Evaluations, and Principal Perceptions of the Process
This report presents the results of a statewide survey of Wisconsin school administrators about their role in the evaluation of teachers.
SREed Brief July 2018 – EE and Job Satisfaction for Novice Teachers
In this brief we examine novice teacher perceptions of their principal and their job and a function of the implementation of their school’s Educator Effectiveness Process.
How are District Differences in the Implementation of Teacher Evaluations associated with Teacher Perceptions of School Leadership and Teacher Job Satisfaction?
We report on the results of a statewide survey of 24,000 teachers—44% of all Wisconsin classroom teachers—about their experiences with teacher evaluations and the feedback process, perceptions of principal effectiveness, and their satisfaction with their job. The results demonstrate a close connection between how school districts implement teacher evaluations and the perceptions of their teachers across a number of school social factors, including the usefulness of performance feedback, principal effectiveness and job satisfaction. Teachers in districts that provide adequate time and support to teachers to complete their evaluation process, along with useful and accurate performance feedback, perceive their principals to be effective leaders and are more satisfied with their job.
Measuring the Effectiveness of Wisconsin Principals: A Study of Wisconsin Framework for Principal Leadership Ratings
This study uses the results of teacher surveys to validate effectiveness ratings assigned to principals as part of the Wisconsin Educator Effectiveness (EE) process. These results suggest ratings reflect important aspects of principal effectiveness and leadership as experienced by teachers and that DPI has developed a process for measuring school leadership that accurately captures the experiences of educators in schools.
Measuring the Principal Effectiveness Equity Gap within Wisconsin School Districts
In this study, we explore the extent that effective principals are equitably distributed across Wisconsin. Within school districts, low-income and diverse schools disproportionally face challenges attracting and retaining effective principals. This results in less effective leadership in these schools. Any efforts to address equity gaps in access to effective teachers must also address equity gaps in access to effective principals. The effectiveness of teachers in diverse and low-income schools is largely determined by the effectiveness of leadership.
Exploring the Perspectives of Milwaukee Opportunity Youth through Photovoice
SREed worked in partnership with the Westlawn Re-engagement Center to conduct a Photovoice evaluation that engaged the voices of Milwaukee Opportunity Youth, young adults ages 16 to 24 who are disconnected from school and work, in developing and refining a pilot program to meet their needs.
Predicting AP Success in the Milwaukee Public Schools
SREed has been working with the Milwaukee Public Schools for a number of years to promote more equitable participation in advanced placement courses. As part of this work, this report summarizes the results of predictive analytics used to identify students with the potential to succeed in AP but who are not participating.
Teacher Job Satisfaction in the Context of the Wisconsin Educator Effectiveness System
This report shows schools that implement EE well have teachers who are more satisfied with their jobs.
Spark i3 Evaluation Study 2
The results of a two-year randomized control trial evaluation of the SPARK literacy model in the Milwaukee Public Schools, funded by an Investing in Innovations (i3) grant, found that it had a positive impact on student literacy, achievement and school attendance.
SPARK i3 Evaluation Study
The results of a two-year randomized control trial evaluation of the SPARK literacy model in the Milwaukee Public Schools, funded by an Investing in Innovations (i3) grant, found that it had a positive impact on reading achievement.