• Intensive Intervention (Part 2): Collecting and Analyzing Data for Data-Based Individualization
Challenge
Initial Thoughts
Wrap Up
Assessment

How can school personnel use data to make instructional decisions?

Page 5: Diagnostic Assessment

In the DBI process, teachers use a diagnostic assessment (Step 3) to determine specific instructional needs. When progress monitoring data indicate that a student is not sufficiently responding to the current intervention, teachers need to determine:

  • Why is the student not responding?
  • What type of change needs to be made to the intervention?

A diagnostic assessment can help teachers to answer these questions. A diagnostic assessment is a tool teachers can use to collect information about a student’s strengths and weaknesses in a skill area. These assessments can be formal (e.g., standardized achievement test) or informal (e.g., work samples).

Listen as Rebecca Zumeta Edmonds discusses several options for conducting a diagnostic assessment (time: 2:08).

Rebecca Zumeta

Rebecca Zumeta Edmonds, PhD
Principal Researcher
American Institutes for Research

View Transcript

Transcript: Rebecca Zumeta, PhD

There’s a variety of tools that schools can use. Depending on their context they may have reasons for choosing one or another. Obviously, when they’re available, standardized diagnostic assessment tools give you the most reliable and valid information. Those may be different kinds of achievement tests that are available in the marketplace. The WIAT is a very common one; the Woodcock Johnson achievement test of some kind, those are a couple of examples. The other options may be doing some more in-depth error analysis of progress monitoring data. The progress monitoring data overall score is not particularly useful for knowing why a kid is or is not making progress. All that tells you is that they’re not making progress. But with many of the measures you can go in and look at the kinds of errors children are making and come up with some hypotheses of about why they may or may not be progressing. For example, with oral reading fluency you can look at the kind of errors that a child is making and get a sense of are they semantic errors, are they errors relating to their understanding of phonics, is the reading very accurate that’s slow and laborious. All of those kinds of patterns tell you different things about the potential area of need that the child may have. And then another type of diagnostic assessment that is much more informal but can help you again come up with a hypotheses about why the student may be struggling to progress would be end-of-unit tests, students’ work samples, homework assignments, things of that nature that really do give some texture to what the child is doing and the skills where they seem to have relative strengths and weaknesses. We realize more and more that as kids get older they’re presenting with academic challenges that’s often co-occurring behaviors, and so we really strongly feel that in any kind of an evaluation for a student or a team getting together and trying to understand why a child may be having a lack of progress that you can’t just look at academics or you just can’t look at behavior in isolation. There really does need to be a broader picture of the child.

Before conducting a diagnostic assessment, it is important to rule out other factors that might be impeding student progress. To do this, the teacher should review a student’s graphed progress monitoring data, looking for patterns that might indicate the reason for the student’s performance. To help determine the reason for a student’s performance, a teacher might find it helpful to consider the questions associated with the student’s data pattern.

Data points near the goal line.

Data Pattern

The student answers the same number of questions correctly even after the intervention has been implemented. This data pattern suggests either:

  • The teacher did not implement the intervention with fidelity.
    x

    fidelity

    In general, this term refers to the degree to which an intervention is implemented accurately, following the guidelines or restrictions of its developers. In this case, fidelity refers to the teacher implementing the adaptation(s) as the instructional decision-making team intended.

  • The intervention was not effective for the student.

Questions

  • Has the intervention been implemented as planned (i.e., with fidelity)?
  • In addition to considering whether the teacher implemented the intervention with fidelity, also consider whether there are other factors that might have affected fidelity (e.g., student absences, field trip, assembly).
 Data points above and below the goal line.

Data Pattern

The student answers questions correctly on some days but not others. He appears to have the knowledge or skills being assessed. This data pattern suggests either:

  • The student was not engaged or motivated.
  • Progress monitoring measures were not administered or scored consistently.

Questions

  • Is the student engaged in or motivated by the intervention?
  • Are there are extenuating circumstances (e.g., family issues, lack of sleep, hunger, not feeling well) that would affect motivation and engagement?
  • Is progress monitoring conducted at the same time and day across weeks?
  • Did the person administering the measure receive adequate training?
  • Is more than one person administering the measure? If so, are they following the same guidelines?
 Data points near the goal line.

Data Pattern

The student answers very few, if any, questions correctly. This data pattern suggests either:

  • The assessment might be too difficult for the student.
  • The intervention was not effective for the student.
  • The progress monitoring probe does not assess the skills being taught.

Question

  • Is the progress monitoring measure targeting the appropriate academic skills?

If school personnel determine that there are factors (e.g., lack of implementation fidelity, lack of student engagement) that may be hindering student progress, these factors need to be addressed. However, if school personnel are able to rule the factors out, the next step is to conduct a diagnostic assessment, which might include an error analysis.

Error Analysis

Boy writing at his desk.Teachers can conduct an error analysis—the process of reviewing error patterns to identify a student’s skill deficits—with progress monitoring data and work samples. An error analysis can help a teacher determine what types of errors were made and why. By identifying error patterns, the teacher can identify a student’s skill deficits and can subsequently align intervention adaptations to the student’s specific needs.

There are a variety of ways to conduct error analyses, and these differ depending on the skills targeted in the intervention and the grade level of the student. Two common areas for which teachers might conduct error analyses are reading and mathematics. Each of these will be discussed on the following pages.

Print Friendly, PDF & Email
Back Next