How can school personnel effectively implement evidence-based practices or programs?

Transcript: Larry Wexler, EdD

Why is fidelity of implementation important, given everything else that’s going on? It is critical for the implementer of a program to implement it as it was researched. There are certain factors that you might want to look at. Perfect example of that is if you have a research-based highly effective literacy program for children six to nine years old, perhaps first to third grade. And you decide you’re going to use that program in a high school for kids essentially without reading skills. It’s unlikely that that program is going to work because you would be implementing it inconsistent with how it was researched.

Length of the sessions is an issue in fidelity. If the program was researched for thirty-minute sessions and you decide you don’t have time to do that so you’ll do it in fifteen-minute sessions, it’s unlikely that you’ll get the same results as the research. Same with the frequency of sessions. If the program was researched to be implemented five times a week and you decide to implement it twice a week, it’s unlikely that it’ll be effective. Another issue to look at in terms of fidelity is qualifications or training of the instructor. If the program was researched using masters-level teachers and you’re going to implement it with all first-year teachers or with student teachers or with parent volunteers, now it may be effective but it’s unlikely that it will work. So fidelity is really critical.

Transcript: Mel Riddile, EdD

Seventy to 80 percent of all change efforts fail, and a big part of that is really the fidelity of that implementation. Are they implementing the program as designed? If we purchased a literacy program, or an after-school tutoring program, or a SAT prep program, are we implementing it as it’s designed to be implemented and intended to be implemented? Are we doing it accurately, and are we doing it consistently? We go in, we do some professional development, and we assume often incorrectly that people are out there doing exactly what we ask them to do. And that’s probably not going to be the case. In fact, I think we need to assume they’re not doing it. In my experience, it’s rare that we are able to have a professional development activity and go into the classroom and find it being implemented properly. So we have to stop assuming that things are done and really work and build in a process that makes sure that we have a number of checks and balances throughout the process to make sure that people understand what we’re trying to do, why we’re trying to do it, and clearly understand what it is we are asking them to do, and they have feedback consistently on are they actually doing it.

Transcript: Joseph Torgesen, PhD

I have a really good example of what happened when a high-quality reading program was not implemented with fidelity, and what happened when the fidelity was improved. This occurred when I first began to work with a principal by the name of Ray King at Hartsfield Elementary School, which had about 70 percent poor children, and they were really struggling in teaching kids to read. Ray was a new principal, and so he came to me for some advice. I said, well, the first thing you have to do is to adopt a common core reading program that implements high-quality research-based instruction and then you need to work with your teachers to see that it’s implemented well across all the kindergarten, first- and second-grade classrooms. Reason I said that is because a good reading program has to be integrated hierarchically—that is, what you do in kindergarten feeds into what you do in first grade, what you do in second grade, and so forth. So Ray selected probably the best currently available reading, early-reading program at that time. then he had lots of conferences with his teachers to encourage them to implement it. He invested quite a bit of money in training and support for them. But there were quite a few teachers on his staff who didn’t want to teach explicitly. They were still of the opinion that kids kind of learned to read on their own as you engage them and made things interesting and exciting, and they particularly didn’t want to teach phonics explicitly, so they basically disregarded those parts of the program. And other teachers really didn’t understand how to teach comprehension strategies, and so they didn’t do that. They had what you call a real spotty implementation of the program. But nevertheless their scores improved somewhat over what they had been the previous year.

We did an analysis of their outcomes at the end of first grade, and we found that this program was implemented weakly, that is, it wasn’t a high-fidelity implementation. They had about 32 percent of their kids who remained poor readers at the end of first grade. Then we looked at that data, and he said there’s lots of room for improvement. Some of the teachers transferred to another school. He got in some new teachers. Other teachers committed to do more of the program than they’d done previously. Then the second year they had a much better implementation, and at the end of that year they had 20 percent of their kids who were still poor readers. So they dropped the number of kids who were still struggling in reading by about 12 percent, simply as a result of implementing this program with greater fidelity. I was always struck by that big jump from the first year of implementation to the second year of implementation being primarily the result of higher fidelity of implementation of the program.