DESIGN PRINCIPLES FOR ACCELERATING STUDENT LEARNING WITH HIGH-IMPACT TUTORING

Carly Robinson | Brown University

Matthew A. Kraft | Brown University

Susanna Loeb | Stanford University

Beth Schueler | University of Virginia

February 2021 | Brief No. 8

Design Principles Series

At A Glance

FREQUENCY

Tutoring is most likely to be effective when delivered in high doses through tutoring programs with three or more sessions per week or intensive, week-long, small-group programs taught by talented teachers

PERSONNEL

Because the skills required for tutoring are different from the skills required for effective classroom teaching, a wide variety of tutors (including volunteers and college students) can successfully improve student outcomes, if they receive adequate training and ongoing support)

MEASUREMENT

Tutoring programs that support data use and ongoing informal assessments allow tutors to more effectively tailor their instruction for individual students.

CURRICULUM

Using high-quality instructional materials that are aligned with classroom content allows tutors to reinforce and support teachers’ classroom instruction.

DELIVERY MODE

Most research has focused on in-person tutoring, but there is emerging evidence that tutoring can be effective when delivered at a distance.

GROUP SIZE

Tutors can effectively instruct up to three or four students at a time. However, moving beyond this number can quickly become small group instruction, which is less personalized and requires a higher degree of skill to do well. One-to-one tutoring is likely most effective but also more costly.

FOCUS

Researchers have found tutoring to be effective at all grade levels—even for high school students who have fallen quite far behind. The evidence is strongest, with the most research available, for reading-focused tutoring for students in early grades (particularly grades K-2) and for math- focused tutoring for older students.

RELATIONSHIPS

Ensuring students have a consistent tutor over time may facilitate positive tutor-student relationships and a stronger understanding of students’ learning needs.

SCHEDULING

Tutoring interventions that are conducted during the school day tend to result in greater learning gains than those that are after-school or during the summer.

PRIORITIZATION

Programs that target lower-performing students can support those students who most need personalized instruction but can also create a negative stigma where tutoring is perceived as a punishment. Programs that target all students in a lower-performing grade level or school benefit from broader organizational commitment and the perception that tutoring is for everyone but are more costly.

The Evidence Base

Rigorous research provides strong evidence that high-impact tutoring can produce large learning gains for a wide range of students, including those who have fallen behind academically.

Tutoring can meaningfully increase learning for a wide variety of K-12 students.

  • A recent meta-analysis reviewed studies of tutoring interventions that have been evaluated by randomized controlled trials in the past few decades and found that, on average, tutoring increased achievement by roughly an additional three to 15 months of learning across grade levels.
  • Another review of almost 200 rigorous studies found that high-impact tutoring—defined as more than three days per week or at a rate of at least 50 hours over 36 weeks—is one of the few school-based interventions with demonstrated large positive effects on both math and reading achievement.
  • While effective tutoring programs can be expensive, their large average effects make them highly cost effective relative to many other educational interventions

Tutoring is one of the most effective ways to increase achievement for students from lower income families.

  • A 2017 study examined interventions that aimed to improve educational achievement for elementary and middle school students from low socioeconomic backgrounds. Of all the interventions examined, including feedback and progress monitoring, cooperative learning, computer-assisted instruction, and mentoring of students, tutoring was most effective.

High-impact tutoring can be scaled and still improve student learning outcomes.

  • Many educational programs that show effects in smaller trials appear less effective when implemented for large groups of students. Large-scale tutoring will not likely replicate the gains found in small-scale studies evaluating tutoring programs under ideal circumstances. However, studies of 15 larger-scale tutoring programs serving between 500 and 7,000 students still found that these programs generated meaningful gains (an average effect size of 0.25 standard deviations).

Although high-impact tutoring is an excellent strategy for addressing COVID-19 learning loss, students most likely to benefit from high-impact tutoring are the least likely to have adequate access without direct school or district action.

COVID-19 is projected to widen inequality in educational outcomes.

  • New research consistently finds evidence that the pandemic and initial school closures increased inequality in educational outcomes across racial and socioeconomic lines, creating an urgency to identify programs and policies—such as high-impact tutoring—to mitigate COVID learning loss.

COVID-19 is likely to widen gaps in access to individualized instruction.

  • The pandemic substantially widened socioeconomic gaps in parents’ searches for online learning resources during spring 2020. Wealthy families are also more likely to hire professional tutors to combat learning loss, exacerbating gaps. Students from lower-income families and schools will likely require additional attention and tutoring resources given lower levels of access to and engagement with supplemental learning tools.

Design Principles

Frequency

Tutoring is most likely to be effective when delivered in high doses through tutoring programs with three or more sessions per week or intensive, week-long, small- group programs taught by talented teachers.

  • Tutoring interventions appear to be more effective as the number of tutoring sessions per week and the overall number of weeks increase.
    • The most effective tutoring interventions involve three or more sessions per week with sessions that last for about 30-60 minutes per day. Studies have found little evidence that once-a-week tutoring is sufficient to generate meaningful gains.
    • Most effective tutoring programs last for at least 10 weeks, but many last for the entire school year.
    • However, elementary students may benefit from shorter but more frequent sessions (i.e. 20 minutes, five times a week).
  • Intensive “vacation academy” programs where small groups of struggling students focus on a single subject over week-long vacation breaks have also generated positive results.
    • With roughly ten-to-one student-teacher ratios, vacation academy programs are more affordable than one-to-one tutoring models. Although they produce meaningful positive effects, the gains are smaller than those observed for state-of-the-art high-impact tutoring.
    • Vacation academies must be carefully designed to ensure impact. Successful programs have drawn on talented teachers who are likely better able to effectively serve small groups than inexperienced volunteers or even average-performing certified teachers. These programs also target students falling below proficiency thresholds but without significant attendance or behavioral issues.

Group Structure

Tutors can effectively instruct up to three or four students at a time. However, moving beyond this number can quickly become small group instruction, which is less personalized and requires a higher degree of skill to do well. One-to-one tutoring is likely most effective but also more costly.

  • Tutoring does not need to be one-on-one.

Personnel

Because the skills required for tutoring are different from the skills required for effective classroom teaching, a wide variety of tutors (including volunteers and college students) can successfully improve student outcomes, if they receive adequate training and ongoing support.

  • Although teachers tend to be the most consistently effective tutors, recent studies have found that AmeriCorps members and paraprofessionals (teaching assistants) can be just as effective when tutoring one-to-one or small groups.
    • One of the most promising studies in recent years used AmeriCorps fellows to conduct daily two-to-one tutoring sessions with low-performing students in 12 Chicago Public Schools. In addition to improved math test scores, students’ math grades improved by 0.58-GPA points, moving students from a C- average to a C+. Students were 50% less likely to fail their math course and 28% less likely to fail a non-math course.
    • Several paraprofessional tutoring programs have proven to effectively increase their scale, including those that deploy service fellows as year-long tutors through AmeriCorps and Saga Education.
  • Successful tutoring programs that rely on volunteers tend to involve intensive training—often at least a couple of weeks—and ongoing support throughout the program, including structured materials and curriculum.
    • Tutoring programs that employ service fellows often require tutors to pass a subject-related exam, as well as undergo trainings focused on instructional techniques, social-emotional learning, and cultural competency.
    • While completely unpaid volunteers have not historically performed well as tutors, “paid volunteers” like paid employees of local businesses whose time is donated and college students on work-study programs show promise.

Focus

Researchers have found tutoring to be effective at all grade levels—even for high school students who have fallen quite far behind. The evidence is strongest, with the most research available, for reading-focused tutoring for students in early grades (particularly grades K-2) and for math-focused tutoring for older students.

  • Reading-focused tutoring interventions for kindergarten and first graders have been evaluated far more often than other tutoring programs.
    • Of 203 studies about the effects of tutoring programs on student learning, 148 assessed the effects on literacy development, the vast majority of which focused on elementary school grades.
  • Fewer studies evaluate tutoring initiatives in middle and high school. At these grade levels, there is greater evidence of success in math outcomes compared to reading outcomes.
    • A study of the Match tutoring program, which provided 9th and 10th grade males in 12 Chicago Public Schools with intensive two-to-one math tutoring as a for-credit class during the school day, found the program reduced math course failures by over 50%, in addition to reducing overall course failures and increasing math grades

Measurement

Tutoring programs that support data use and ongoing informal assessments allow tutors to more effectively tailor their instruction for individual students.

  • Tutoring programs that effectively use data are more likely to be successful.
    • Many of the most successful tutoring interventions to date collected ongoing implementation data and used it to continuously improve their programs.
    • Successful tutoring organizations often have clear learning objectives and engage in rigorous evaluations to assess the effectiveness of their program.
  • Frequent assessments of learning allow tutors to personalize instruction based on individual students’ needs.
    • Formative assessments, whether conducted in the classroom or in a tutoring session, provide the tutor with timely feedback on each student and allow for personalized instruction.
    • The Reading Partners program monitors student progress using mid-year and end-of-year assessments and updates each student’s reading plan after the mid-year assessment.
    • For formative assessments to result in more student learning, tutors need time and support to review the assessment, as well as knowledge of how to address each student’s needs.
    • Minnesota Math Corps tutors are provided with regular professional development to improve their data-based decision-making

Relationships

Ensuring students have a consistent tutor over time may facilitate positive tutor- student relationships and a stronger understanding of students’ learning needs.

  • Tutoring programs that pair a student with a consistent tutor for the duration of the program may provide better learning environments.
    • Many tutoring interventions keep one tutor with a set student or group of students for the duration of the program.
    • Tutors that meet regularly with students have the opportunity to get to know individual students’ strengths and weaknesses, while developing caring tutor-student relationships.
  • Like positive teacher-student relationships, positive tutor-student relationships may lead to greater academic, social, and motivational outcomes.
    • Some successful tutoring interventions are relationship-driven programs that focus on building trusting relationships between tutors and students, in addition to enhancing academic achievement.
    • When students have positive, caring relationships with their tutors, they may become more motivated and have better attitudes toward school. Research on youth mentoring programs suggest positive mentoring relationships can have a wide range of benefits for students’ social-emotional well-being.
    • Small-group vacation academy participants who were assigned to the same teacher for the week-long program saw the largest reductions in suspensions after the program.

Curriculum

Using high-quality instructional materials that are aligned with classroom content allows tutors to reinforce and support teachers’ classroom instruction.

  • Students likely learn more when their tutoring sessions complement and are responsive to their classroom grade-level instruction.
    • High-impact programs tend to use high-quality instructional materials that align with their core classroom content or grade-level standards.
    • Remediation, or giving students simpler, previous grade-level materials, had been found to result only in students falling further behind the grade-level material.
    • Rather than focusing on items that students have failed to master previously, tutors can address missed concepts and skills that are most critical to accessing the upcoming content.
    • Tutors can take on the role of catching struggling students up, while taking some of the pressure off classroom teachers and allowing them to maintain the pace of their instruction. However, tutors need to ensure they build foundational skills while making connections to the content students are learning in class.
    • There is some suggestive evidence that tutoring programs that employ school-based coordinators who connect tutors to teachers may be more successful.

Scheduling

Tutoring interventions that are conducted during the school day tend to result in greater learning gains than those that are after school or during the summer.

  • The recent meta-analysis of tutoring studies found that the effects of programs conducted during the school day are roughly twice as large as those conducted outside of school.
    • Providing tutoring during school or immediately after school increases the likelihood that students will attend their tutoring sessions and facilitates a more academic-focused culture.
    • Tutoring that was a feature of the No Child Left Behind supplementary education services was delivered by independent providers outside of the school day. Poor student attendance and a disconnect from students’ school experiences often led to disappointing results.

Delivery Mode

Most research has focused on in-person tutoring, but there is emerging evidence that tutoring can be effective when delivered at a distance.

  • Virtual learning offers the potential to lower tutoring costs and potentially provide stronger matches between students’ needs and tutors’ skill sets by expanding the potential supply of tutors.
    • Preliminary results from a recent evaluation of the Saga Education tutoring program in Chicago and New York City public schools found that a blended model was equally effective at increasing student learning as the costlier in-person only tutoring.
    • A recent small-scale evaluation of an elementary math online tutoring program found promising results. Students who received online one-to-one tutoring showed greater gains on a math assessment than those who did not, which compares favorably to effects found by other, in-person elementary math tutoring programs.

Prioritization

Programs that target lower-performing students can support those students who most need personalized instruction but can also create a negative stigma where tutoring is perceived as a punishment. Programs that target all students in a lower- performing grade level or school benefit from broader organizational commitment and the perception that tutoring is for everyone but are more costly.

  • Decisions about which students to target are central; however, several different models have led to positive effects.
    • Need-driven: Most tutoring interventions that have undergone evaluation have been need-driven and targeted to students who perform below particular thresholds. In 19 schools across three states, Reading Partners identified more than 1,100 students who were half a year to two and a half years behind grade level in reading to receive intensive reading tutoring weekly for 28 weeks on average.
    • Curriculum-driven: Tutoring can also be curriculum-driven and provided at critical stages of students’ learning development or school transitions, points at which the disruption can negatively affect student achievement. For instance, Reading Recovery tutoring programs occur during first grade because it is a crucial point for literacy development.
    • Universal tutoring: The Match tutorial model, where all students in a grade or school receive tutoring, may make tutored students feel less stigmatized, could address the needs of the mid-performing students, and may provide a vehicle for high-performing students in underserved communities to excel. Districts or states can also target low-performing schools but provide tutoring schoolwide to address equity concerns without generating stigma for individual students.
  • Framing matters for avoiding negative stigma.
    • Vacation academies in Lawrence, Massachusetts were pitched as a “special opportunity” rather than remediation and achieved high student engagement with student prizes and interspersed recreation. These programs were one part of broader “turnaround” policy efforts and therefore were implemented in some of the state’s lowest-performing schools

Caveats & Considerations

Tutoring programs may exacerbate educational inequalities if schools do not take an active role in connecting students who could benefit from tutoring with tutors.

  • Many students who would benefit from tutoring are unlikely to sign up for tutoring on their own.
    • Schools can reduce barriers for students by actively making tutoring a core part of students’ educational experience.

It is much harder for tutors to deliver effective individualized instruction if they have to tutor more than three or four students at a time, especially if tutors are not experienced educators.

  • While tutoring can involve up to four students with one tutor, teachers might be better able to tutor up to four students whereas paraprofessionals and volunteers may do better with one or two students at a time.
    • Paraprofessionals and volunteers may be better suited to one-to-one tutoring because they are less likely to have developed the skills in behavior management and group instruction that are needed for working with multiple students.

Tutoring is not a silver bullet. Students are unlikely to experience learning gains if schools do not commit time and resources to implementing a high-quality tutoring program well.

  • Studies have found little evidence that once-a-week tutoring is sufficient to generate meaningful effects.
    • One meta-analysis found that high-dosage tutoring was 20 times more effective than low-dosage tutoring in math. In reading, high-dosage tutoring was 15 times more effective than low-dosage tutoring.
  • Simply providing students with access to tutoring is unlikely to be effective for all students.
    • Tutoring interventions often are not successful when there are no minimum dosage requirements, little oversight, and minimal connections with the students’ schools. A key element of successful tutoring programs is being able to establish a rigorous and caring culture


This EdResearch for Action Project brief is a collaboration among:

Funding for this research was provided by the Bill & Melinda Gates Foundation. The findings and conclusions contained within are those of the authors and do not necessarily reflect positions or policies of the foundation.

More Like This