• Evidence-Based Practices (Part 3): Evaluating Learner Outcomes and Fidelity
Challenge
Initial Thoughts
Perspectives & Resources

What is the process for evaluating the effectiveness of an EBP with your children or students?

How do you measure infant, child, or student performance?

How do you know whether you are correctly implementing an EBP?

How do you know whether an EBP is effective with your children or students?

Resources

Wrap Up

Evidence-Based Practices (Part 3): Evaluating Learner Outcomes and Fidelity

Assessment

Take some time now to answer the following questions. Please note that the IRIS Center does not collect your Assessment responses. If this is a course assignment, you should turn them in to your professor using whatever method he or she requires. If you have trouble answering any of the questions, go back and review the Perspectives & Resources pages in this module.

  1. Why is it important to monitor fidelity of implementation?
  1. Explain why it is important to review both fidelity of implementation data and progress monitoring data to determine the effectiveness of an EBP for a students.
  2. View the progress monitoring data graphs below.
    1. Discuss the performance of the student in Graph A.
    2. Discuss the performance of the student in Graph B.
    3. Describe the instructional decisions that can be made based on the data.

    Graph A: EBP across 14 weeks, starting with an acceptable level of fidelity at Week 2, dipping below at Week 3, and hovering slightly below or above the goal line from Weeks 4 to 14.Graph B: EBP across 14 weeks, starting with an acceptable level of fidelity at Week 2, dipping below at Week 3, and hovering slightly below or significantly below the goal line from Weeks 4 to 14.

  1. Discuss two ways to collect observational data on fidelity of implementation. Explain why one method is better than the other.
  1. The movie below shows a teacher implementing the POW writing strategy.
    1. Print the POW Observation Checklist.
    2. While watching the movie, use the checklist to record your observational data.
    3. Calculate the teacher’s fidelity of implementation.

    Transcript: Video

    Teacher: Who can tell me what makes a good story? Will?

    Will: Well, if it has a great action scene in like the very middle of it.

    Teacher: A great action scene. Good.

    Student: A hook that could, like, get you into the book before you quit it.

    Teacher: So you have kind of the reader that wants to be sucked into the story and be aware. Very good. Any other ideas on what makes a good story? Stuart?

    Stuart: Make the story kind of make sense.

    Teacher: The story needs to make sense. Have you ever read a story that you just read it and said, “What was that about? I have no idea. It made no sense at all!” You want a story to make sense. Very good. Do stories have to be true?

    Class: No.

    Teacher: No. What kind of stories can you have? Sarah?

    Sarah: Fantasy.

    Teacher: Huh?

    Sarah: Fantasy stories.

    Teacher: Fantasy stories. Victor?

    Victor: Fiction.

    Teacher: Fiction. Ellie?

    Ellie: Biography.

    Teacher: Biography. Elias?

    Elias: Um, Greek mythology.

    Teacher: Greek mythology. Cameron?

    Cameron: Maybe a novel?

    Teacher: A novel. So, yeah, you have all different kinds of stories, and we want stories to have all their parts and make sense. We want them to be fun to read. We want to be able to hook our readers, and they need to have all their parts.

    Teacher: Today, we’re gonna learn about how to use it with story writing, ok? So the “P” in POW is “pick my idea.” We need to pick an idea. What’s my story gonna be about. What can a story be about?

    Teacher: The second part of the trick is organize my notes. Does anybody write out any notes when they make a story? You do? Do you write down what all ideas of how you’re gonna write your story? Yeah, it helps me a lot when I have to write a story, ’cause I can’t keep all my ideas in my head and sometimes I forget them, and if I don’t write them down, I won’t have them later when I go to write. So I am gonna teach you a trick on how to organize your notes for stories. Ok? And then, the third part, is write and say more. So I’m gonna take my notes, and I’m gonna write my story and I’m gonna add more to it.

    Teacher: So today I’m going to share with you how I write, ‘cause sometimes I have trouble writing. And I’m going to show you how I use my tricks. So I have this picture here, and now I have to think, “Hhhhmmm. What am I going to write a story about? What do I do first? This is hard. I know! I’ll use POW.”

    Teacher: So first I have to…What’s “P”?: “Pick my idea.” I’m going to pick an idea. Okay, so he’s looking at a magnifying glass. It’s kind of a silly picture. I really don’t like it. Okay, keep my mind free. I can think of a good idea. I think my idea’s going to be…he’s going on a big adventure. He’s searching for a treasure. That’s what my idea is going to be. So I’m doing POW, okay? So I picked my idea. He’s going on a treasure hunt. All right, that’s done.

    Teacher: Okay, so next I have to “Organize my notes.” Writing is so tough. All right, what do I do to organize my notes? That’s right, I learned a trick. W-W-W, What = 2, How = 2. Okay, so I have this. This is my graphic organizer. This will help me remember my parts. So WWW. “Who?” Oh, I don’t know that one. I can skip that one. “When?” He looks kind of modern, but he has a canteen and a magnifying glass. I’m going to say, like, 1903. Okay. “Where?” Where would somebody go on a treasure hunt? Um, I have no idea. Who does treasure hunts anymore? Okay, keep my mind free. I can do this. Where? Ooh, the desert in Africa. That’ll be good. I like that idea! “Desert.” “Africa.” Okay.

    Teacher: That’s it. I have all my parts. I organized my notes. Yes…alright, I picked my idea, I organized my notes, and now I’m going to write and say more. So now I’m gonna use my notes, so I’m gonna say 1903, I’m gonna check that off, ’cause I have that part done. Okay so you guys are sitting in pairs, and you’re going to work with your partner on memorizing the trick. Sometimes you might want to write a story at home or if you keep a journal at home, or if you go on an airplane, and go far far away, and decide to write a story about your trip. You’ll want to use the trick, and you won’t always have the poster to help you out with the trick. So if you memorize it, then you can take it with you and use it anywhere, and use it all the time. Okay so you can turn to your partner and you have these little cards, and you can quiz each other to help you memorize the trick. All right so today we’re gonna write a story together, and I’m going to need your help. I have this picture here. So what am I going to do first? Victor…

    Victor: Who’s the main character?

    Teacher: Who’s the main character? That’s good, but what am I going to do before that?

    Student: Pick your idea.

    Teacher: Pick my idea. What’s the trick I use for all kinds of writing?

    Students: POW.

    Teacher: So I am going to pick my idea. What’s my idea going to be? Thank you. Anybody have an idea?

    Thomas: They’re going on a family road trip.

    Teacher: Family road trip. Ok, so I’ve picked my idea. So who’s my main character going to be on my family trip?

    Student: Fat guy driving. (Children laughing.)

    Teacher: I’m going to put “guy driving.” So do you notice I don’t write complete sentences?

    Teacher: We wrote and said more. This is the story we wrote based off of our notes from the graphic organizer.

    Teacher: Okay, so you guys are going to practice working on these tricks by yourself, okay? You each have a picture on your table, and you have a graphic organizer just like the one that I’ve been using on the board. But you have a smaller one. So I want you to work on and think about using POW and W-W-W, What = 2, How = 2 to write a story about those pictures, okay? If you have any questions, raise your hands. I’ll be walking around to help you.

    (Close this panel)

    Print Friendly, PDF & Email
    Back