How do you know whether you are correctly implementing an EBP?
Page 5: Identifying a Fidelity Measure
Recall that fidelity of implementation means that you are implementing an EBP as intended by the researchers or developers. Fidelity of implementation consists of key components: adherence, exposure/duration, and quality of delivery. Implementing an EBP with fidelity increases the likelihood that your young children or students will experience the intended outcomes.
Key Components of Fidelity of Implementation
Adherence—Following the instructional procedures of the practice or program as they were intended and implementing all components of the EBP in the correct order
Exposure/duration—Implementing the practice or program for the recommended:
- Length of session (e.g., 40 minutes)
- Duration of EBP (e.g., 12 weeks, one semester, one academic year)
- Frequency (e.g., daily, three times per week)
Quality of delivery—Delivering the EBP using good teacher practices (e.g., implementing with enthusiasm, providing time for student questions and feedback, managing transitions well)
To learn more about implementing an EBP with fidelity, view the IRIS Module:
To determine whether an EBP is effective with your children or students, your fidelity of implementation must be high. To monitor fidelity, you should collect data on how well you are implementing the EBP. But first you must decide how you are going to collect data. Research on implementation fidelity has demonstrated that observations are the best method for monitoring fidelity of implementation. You can accurately and systematically collect observational data by using a checklist. This checklist, or observational form, might be included with a practice or program. If not, you should develop your own. To develop your own observational checklist, use the simple two-step procedure below.
Step 1: Conduct a task analysis of the EBP procedures. A task analysis is the process of breaking down activities into smaller, sequenced steps that can be easily defined and readily observed.
Below is a sample task analysis of Peer-Assisted Learning Strategies (PALS) for grades 2–6. The four major components or activities of PALS 2–6 (i.e., Partner Reading, Retell, Paragraph Shrinking, Prediction Relay) are listed and then broken down into smaller steps. Additionally, other key elements of the procedures are also listed.
Step-by-Step Procedures
General Procedures
- Introduce PALS session
- Have students transition to pairs quickly and quietly
- Have system in place for students to quickly and easily access materials
Specific Procedures
- Implement Partner Reading with Retell for 12 minutes
- Follow script
- Have students switch jobs after 5 minutes
- Instruct lower performing reader to retell story for 2 minutes
- Monitor students
- Implement Paragraph Shrinking for 10 minutes
- Follow script
- Have students switch jobs after 5 minutes
- Monitor students
- Implement Prediction Relay for 10 minutes
- Follow script
- Have students switch jobs after 5 minutes
- Monitor students
- Have students put away materials in 2 minutes or less
Step 2: Develop an observational checklist. Two types of items should be included on the checklist. The first addresses the number of procedural steps to which educators should adhere, which correspond to the items in the task analysis. The second addresses the quality of implementation—how well the educator implements the intervention or instructional procedures. To more objectively rate the quality items, it is beneficial to define what each point on the scale represents (e.g., very well=instructed with enthusiasm).
Although there are many ways to create fidelity checklists for PALS 2–6, one option is illustrated below. The checklist should include all of the items in the task analysis (included at the bottom of this page). These items will capture how well the teacher is following the implementation procedures. Additionally, the checklist should include at least one or two items that address quality of implementation, such as those in the boxes in the checklist below.
This form can be used to check the fidelity of implementation during one PALS session, allowing the observer to collect data for multiple components of fidelity (i.e., adherence, length of session, quality of delivery). Although not applicable here, data on the other components of fidelity (i.e., number of sessions, frequency of implementation, child/student responsiveness) should also be considered when evaluating implementation fidelity
View the PALS 2—6 Observation form as a PDF
Task Analysis for PALS 2—6
PALS 2–6
Step-by-Step Procedures
General Procedures
- Introduce PALS session
- Have students transition to pairs quickly and quietly
- Have system in place for students to quickly and easily access materials
Specific Procedures
- Implement Partner Reading with Retell for 12 minutes
- Follow script
- Have students switch jobs after 5 minutes
- Instruct lower performing reader to retell story for 2 minutes
- Monitor students
- Implement Paragraph Shrinking for 10 minutes
- Follow script
- Have students switch jobs after 5 minutes
- Monitor students
- Implement Prediction Relay for 10 minutes
- Follow script
- Have students switch jobs after 5 minutes
- Monitor students
- Have students put away materials in 2 minutes or less
Click here to view a sample fidelity checklist for dialogic reading*, a reading strategy often used in early childhood settings.
Listen as Sam Odom offers more information on monitoring fidelity of implementation (time: 1:34).
Sam Odom, PhD
Professor, Special Education
Director, Frank Porter Graham Child Development Institute
University of North Carolina at Chapel Hill
Transcript: Sam Odom, PhD
Evidence-based practice is only evidence-based to the extent that it’s implemented with fidelity. The practice should have an assessment that identifies the steps that are necessary for a practitioner to follow in order to use the practice correctly. Often practices have checklists. They’re almost like a task-analysis of what you do in a session or how you might conduct a specific practice. So that’s one step: having the practice clearly described and articulated, having a way for the practitioner to determine if they’ve actually followed the steps in the practice and used it appropriately, receiving feedback about whether the practitioners are using the practice in a specific way. And often that feedback is preceded by some form of training, which might be a workshop or through self-instruction. Both of those—workshops and accessing information in a self-learning way—I think, might be good introductory steps to learning a practice. Many people talk about how workshops really don’t lead to use of a practice with fidelity without some additional coaching or support. It could be an expert or a consultant who provides the coaching or supervisor. It could be a peer who just gives feedback on a practitioner using a strategy in a specific and correct way.
*CONNECT (2011). Dialogic Reading Observation Form. Chapel Hill: University of North Carolina, FPG Child Development Institute, Author. The contents of this form were developed under grants from the US Department of Education, #H325J070007 and Doing What Works. However, those content do not necessarily represent the policy of the US Department of Education, and you should not assume endorsement by the Federal Government.
