How can school personnel effectively implement evidence-based practices or programs?
Page 4: Understand Fidelity of Implementation and Its Importance
Discovering what works does not solve the problem of program effectiveness. Once models and best practices are identified, practitioners are faced with the challenge of implementing programs properly. A poorly implemented program can lead to failure as easily as a poorly designed one.
Mihalic, Irwin, Fagan, Ballard, & Elliot, 2004
Once school personnel have selected an evidence-based practice or program, they are ready to begin the implementation process. When a practice or program is implemented as intended by the researchers or developers, this is referred to as fidelity of implementation. Other commonly used terms include treatment integrity, procedural fidelity, intervention integrity, procedural reliability, and procedural adherence. Generally, to implement a practice or program with fidelity, an individual should:
Be adequately trained
Adhere to the instructional procedures of the practice or program (e.g., follow the script, implement among groups of the correct size)
Implement the practice or program as frequently as recommended (e.g., daily, three times per week)
Implement the practice or program for the recommended amount of time (e.g., one semester, one academic year)
Skillfully implement the instructional procedures
Unfortunately, research indicates that, though many schools are implementing evidence-based practices, they often do not implement them as intended.
Lack of implementation fidelity might result in a practice or program being less effective, less efficient, or producing less predictable responses. (Wilder, Atwell, & Wine, 2006; Noell, Gresham, & Gansle, 2002)
When programs implemented with fidelity are compared to programs not implemented with fidelity, the difference in effectiveness is profound. Those implemented with fidelity yield average effect sizes that are two to three times higher. (Durlak & DuPre, 2008)
To hear experts discuss the importance of fidelity of implementation, click on each panelist below.
Larry Wexler, Ed DU.S. Department of Education Office of Special Education Programs
Why is fidelity of implementation important, given everything else that’s going on? It is critical for the implementer of a program to implement it as it was researched. There are certain factors that you might want to look at. Perfect example of that is if you have a research-based highly effective literacy program for children six to nine years old, perhaps first to third grade. And you decide you’re going to use that program in a high school for kids essentially without reading skills. It’s unlikely that that program is going to work because you would be implementing it inconsistent with how it was researched.
Length of the sessions is an issue in fidelity. If the program was researched for thirty-minute sessions and you decide you don’t have time to do that so you’ll do it in fifteen-minute sessions, it’s unlikely that you’ll get the same results as the research. Same with the frequency of sessions. If the program was researched to be implemented five times a week and you decide to implement it twice a week, it’s unlikely that it’ll be effective. Another issue to look at in terms of fidelity is qualifications or training of the instructor. If the program was researched using masters-level teachers and you’re going to implement it with all first-year teachers or with student teachers or with parent volunteers, now it may be effective but it’s unlikely that it will work. So fidelity is really critical.
Transcript: Mel Riddile, EdD
Seventy to 80 percent of all change efforts fail, and a big part of that is really the fidelity of that implementation. Are they implementing the program as designed? If we purchased a literacy program, or an after-school tutoring program, or a SAT prep program, are we implementing it as it’s designed to be implemented and intended to be implemented? Are we doing it accurately, and are we doing it consistently? We go in, we do some professional development, and we assume often incorrectly that people are out there doing exactly what we ask them to do. And that’s probably not going to be the case. In fact, I think we need to assume they’re not doing it. In my experience, it’s rare that we are able to have a professional development activity and go into the classroom and find it being implemented properly. So we have to stop assuming that things are done and really work and build in a process that makes sure that we have a number of checks and balances throughout the process to make sure that people understand what we’re trying to do, why we’re trying to do it, and clearly understand what it is we are asking them to do, and they have feedback consistently on are they actually doing it.
Transcript: Joseph Torgesen, PhD
I have a really good example of what happened when a high-quality reading program was not implemented with fidelity, and what happened when the fidelity was improved. This occurred when I first began to work with a principal by the name of Ray King at Hartsfield Elementary School, which had about 70 percent poor children, and they were really struggling in teaching kids to read. Ray was a new principal, and so he came to me for some advice. I said, well, the first thing you have to do is to adopt a common core reading program that implements high-quality research-based instruction and then you need to work with your teachers to see that it’s implemented well across all the kindergarten, first- and second-grade classrooms. Reason I said that is because a good reading program has to be integrated hierarchically—that is, what you do in kindergarten feeds into what you do in first grade, what you do in second grade, and so forth. So Ray selected probably the best currently available reading, early-reading program at that time. then he had lots of conferences with his teachers to encourage them to implement it. He invested quite a bit of money in training and support for them. But there were quite a few teachers on his staff who didn’t want to teach explicitly. They were still of the opinion that kids kind of learned to read on their own as you engage them and made things interesting and exciting, and they particularly didn’t want to teach phonics explicitly, so they basically disregarded those parts of the program. And other teachers really didn’t understand how to teach comprehension strategies, and so they didn’t do that. They had what you call a real spotty implementation of the program. But nevertheless their scores improved somewhat over what they had been the previous year.
We did an analysis of their outcomes at the end of first grade, and we found that this program was implemented weakly, that is, it wasn’t a high-fidelity implementation. They had about 32 percent of their kids who remained poor readers at the end of first grade. Then we looked at that data, and he said there’s lots of room for improvement. Some of the teachers transferred to another school. He got in some new teachers. Other teachers committed to do more of the program than they’d done previously. Then the second year they had a much better implementation, and at the end of that year they had 20 percent of their kids who were still poor readers. So they dropped the number of kids who were still struggling in reading by about 12 percent, simply as a result of implementing this program with greater fidelity. I was always struck by that big jump from the first year of implementation to the second year of implementation being primarily the result of higher fidelity of implementation of the program.
Promoting Fidelity of Implementation
In addition to school personnel implementing an evidence-based practice or program as intended, there are a number of other factors that are often associated with high fidelity of implementation. By attending to such factors––those related to the practice or program, the organization, the teacher, and training––school personnel are more likely to implement the practice or program with fidelity, which in turn increases the possibility of positive student outcomes.
Factors Associated with High Implementation Fidelity
Practice or Program Characteristics
Addresses students’ needs
Has been shown to be effective with similar student populations
Requires few resources
Accompanied by a clear and comprehensive instructional manual
Clear roles and responsibilities
Shared decision making
Perceived need for the practice or program
Acceptance and belief in the practice or program
Having or acquiring necessary skills to implement the practice or program
High-quality professional development
Available technical assistance
With these factors in mind, there are actions that school personnel can take when implementing a new practice or program. These actions, listed below, can increase the likelihood that education professionals will implement the new practice or program with high fidelity.
Establish an implementation team
Train staff on how to implement the practice or program
Provide ongoing training and support for the implementation of the practice or program
Use existing manual(s) or create clear guidelines for the implementation process
Monitor implementation fidelity
The following Perspectives and Resources pages will explore these actions in greater detail.