The IRIS Center attempts to gauge the effectiveness of our resources and materials. Below, you will find a report about a pair of studies designed to determine whether the use of our modules improved the outcomes for groups of college students primarily composed of general and special education majors.
During the 2004–2005 and 2005–2006 academic years, a study of the learning outcomes associated with the use of an IRIS Module was conducted at a large urban campus with a diverse student body. The module’s content pertained to instructional accommodations for students with visual disabilities. In an introductory class aimed primarily at general and special education majors, and with multiple sections offered each quarter, three instructors taught the course one quarter without the IRIS Module, and the next quarter with the IRIS Module. A total of 620 students were involved across these sections and two conditions.
Students in both conditions were administered a pretest and post-test. These involved a hypothetical scenario in which a teacher learns that two students with visual disabilities will be among her incoming class. Takers of the pretest and post-test were instructed to indicate what the teacher should do, and their responses were scored. Responding to this scenario involved applying to the scenario content that was covered by the text and/or the module. In every case, the average post-test score for students who viewed the module was significantly higher than that for students who did not (see Figure A, below). The average effect size—an indicator of the magnitude of the difference between the two conditions—was 1.08, which can be interpreted as a large effect.
Because the module contained content that was common to both the module and the text and content that was unique to the module, we also looked at students’ responses according to what content was reflected (common versus unique). In terms of how much they changed from the pretest to the post-test, positive gains in learning occurred for both types of content. Further, for two of the three instructors, the gains exhibited were significantly larger for students who viewed the module with respect to both types of content (see Figure B, below). This suggests that the effectiveness of modules may be in both reinforcing material presented in textbooks and infusing this material with new and additional content.
A second study of the impact of IRIS Modules on student learning was conducted to examine how factors related to instructor application of the principles of the How People Learn theory (HPL) affect student learning. The module’s content pertained to teaching self-regulation strategies to students. In an introductory class at a large state university aimed primarily at general and special education majors, and with multiple sections offered each semester, three instructors taught the course one semester using the IRIS Module as an independent assignment, considered low HPL (Independently Viewed condition) and the next semester by including portions of the IRIS Module in one of their lectures and then leading a discussion on the module’s Initial Thoughts questions, considered high HPL (Instructor Enhanced condition).
A total of 480 students were involved across these sections and two conditions. Students in each condition were administered a pretest and post-test, which were composed of both multiple-choice (or closed-ended) and free-response (or open-ended) questions and which presented several scenarios related to teaching self-regulation strategies to students. Responses to these questions involved the application of content covered by the module.
Figure C displays the mean total scores on the pretest and post-test for both the Independently Viewed and Instructor-Enhanced conditions. As is obvious from the figure, the pretest and posttest standings for the two conditions were not statistically different. In other words, students learned the content equally well whether they completed the module independently as homework or during an instructor-led session in class.
Within each condition, the gain in factual knowledge was statistically significant. However, students in both conditions achieved similar gains. Thus, it appears that, although students did gain in their factual knowledge about self-regulation, more involvement by the instructor did not result in enhanced performance.
Also shown in Figure C is the average performance on factual knowledge for the 138 students enrolled in two sections of the same course for the following semester. These individuals did not view the module but rather completed the post-test measures at the end of the semester. Each of the module conditions differs significantly from this control condition. These results add to the evidence that students who worked through the module improved their factual knowledge about self-regulation. The average gains in factual knowledge were similar across all instructors and conditions.
Ability to Apply Knowledge to Clearly Defined Situations
Five questions in the outcome measure addressed students’ ability to identify instructional situations in which self-regulation was an appropriate strategy. Presented with a scenario in which a hypothetical student experiences difficulty in learning, test-takers were first asked to select the strategy best suited to correcting the problem. Self-regulation was, of course, the correct response; however, another question added a bit more detail, informing test-takers that the hypothetical student showed variability in performance. The purpose of this was to provide test-takers with additional information that would more clearly identify self-regulation as a useful response. The remaining questions asked the test-takers to select from several situations: (1) two situations that would best be helped by self-regulation and (2) two situations that would least be helped by this strategy. They also were asked to provide the rationale for their choice, hopefully citing the key features of self-regulation.
Figure D shows how students in each condition (Independently Viewed condition and the Instructor-Enhanced condition) scored on the pretest and the post-test and how students in the posttest-only group performed. Students in the Control Group performed significantly lower than both module groups and these results did not differ by instructor.
Similar to gaining more factual knowledge, students in both conditions also significantly increased in their capacity for identifying whether self-regulation was appropriate for specific types of situations. Again, however, there was no differential increase in this learning outcome between conditions.
The difference between the two module conditions was significantly different with the Viewed Independently group doing somewhat less well on the post-test. Although it was the case that the average post-test score was significantly higher for students in the Instructor-enhanced condition, the gain in learning for both conditions was the same. That is, the difference in post-test scores between the Independently Viewed condition and the Instructor-Enhanced condition was primarily attributable to the fact that students in the former group scored lower on the pretest (see the parallel lines in Figure D). This pattern was evident for each of the three instructors.