How can faculty deliver and refine their online courses?

Now that your online course is up and running, you should monitor your teaching with an eye toward making improvements. Fortunately, online courses allow you to build in tools and collect data to evaluate the strengths and weaknesses of your design. Following are four ways to do so. (Note that the first three have to be intentionally included in the design so that the data will be available for use in subsequent analyses.)

• Item analysis for quizzes and rubrics. Item analysis is the process of identifying patterns in student performance on assessments and then determining what those patterns say about the assessment instruments. Item analysis can be a good way to start the course-revision process. In some LMSs, statistics about student performance are built into the assessment tools and learning activities. Click here for examples.

  Examples

  Example 1: The course LMS indicates that most students, including your highest-performing students, got one particular quiz question wrong. As an instructor, you can now use that information to determine whether the content of that question was explored deeply enough, or perhaps students did not have an opportunity to thoroughly process that content, or maybe the question was just poorly written. Now you can make changes accordingly.

  Example 2: You assign a major project and use a rubric for grading. The rubric statistics indicate that almost all students in the course met or exceeded expectations for the criteria called “Methodology Section.” However, only 5% of students met or exceeded expectations on “Plan Quality,” which indicates a gap in instruction. Based on this item analysis, you recognize the issue and give students another opportunity to submit their plan with a more detailed rubric to guide their work. You also adjust your module structure for future iterations of the course to include more opportunities for discussing and getting peer feedback on the plan before submitting it.

• Student feedback. Although student feedback on end-of-term course evaluations can be useful for revising a course for the future, feedback during the semester will allow you to make changes that will benefit your current students. Following are several ways to collect this type of feedback.

  • Informal mid-semester survey
  • Non-graded, reflective question
  • Exit ticket

  Informal mid-semester survey

  This type of survey, modeled after the end-of-term course evaluation, can be used to collect feedback on the course, content, and instructor performance one or more times per semester. Your LMS might contain survey tools that can be used for this purpose.

  Non-graded, reflective question

  You can add an optional and voluntary question about learning into any assessment (e.g., “What else did you learn in this unit that I haven’t asked about on this test?” or “What learning experiences were most helpful in preparing you for this assessment?”).

  Exit ticket

  At the end of class, ask the students a question that can be answered relatively quickly to assess their learning. For an online class, this could be in the form of an online poll or a discussion board post.

• Click Tracking. In some LMSs, instructors can collect data on the course links that students are utilizing. Depending on your LMS, this feature may be automatic or it may need to be turned on or enabled. This allows instructors and students to see what materials students have clicked on and when. If students are not responding to certain topics or types of activities at the same level as others, you may want to rethink those learning opportunities, provide more scaffolds, or review their alignment with course goals or learning objectives.

• Instructor Field Notes. Taking notes about your experiences while they are still fresh can help you make data-informed decisions about adjustments to your future course design. To get started, record information about the following:

  • Amount of time you spent preparing and grading
  • Items or activities that were hard, confusing, or just not worth the effort
  • Where you felt time was wasted and how to make teaching this course more efficient
  • Items or activities that the students complained about, experienced difficulty, or needed support
  • Elements that were fun, as well as those that were not
  • Whether students demonstrated that they met the course goals and module learning objectives

A frequent challenge of online teaching is maintaining a high level of teacher-student engagement. As you teach your course, be sure to keep close track of the ways in which you are available for and interact with your students. Below are some suggested ways to maintain or increase the amount of teacher presence in your course.

• Consistently check into the interactive asynchronous parts of your course. If you didn’t schedule a time to respond to discussion board posts prior to the beginning of class, be sure to set a consistent time to do so.
• Respond to student emails. Again, if you didn’t communicate a plan for responding to emails prior to or at the beginning of the course, it is recommended that you do so.
• Provide timely, focused, personalized feedback on student work. Offering quick feedback will help prevent your students from feeling they are alone in the course. To speed up how quickly you can offer feedback, you may want to:
  • Limit feedback to areas directly connected to course learning goals.
  • Use video and audio feedback tools to speak directly to students as you review their work.
  • Pre-load common feedback into a rubric so all you have to do is click on the correct level, and students will get your feedback automatically.
  • Offer feedback in two phases. As soon as work is submitted, offer a short response with general comments. Later, when time allows, provide a more detailed evaluation and notes. If you use this strategy, make sure students understand that feedback comes in two phases and what they should expect in each.

• Facilitate focused synchronous meetings. If the synchronous portion of your course includes video conferencing:
  • Make sure you are familiar with the controls on your video conferencing platform and be prepared to use the security features in the event that uninvited or misbehaving participants join your meeting.
  • Prior to the meeting, provide an agenda and guidelines for students related to expectations for participation.
• Make sure students stay engaged. Class progress tools (e.g., Intelligent Agent in Brightspace) can help automate some of the labor of tracking student engagement. A quick scan of the data provided by the tool can provide useful information about which students are not engaging with the content or other tools. However, there is no substitute for old-fashioned human contact. Holding office hours and reaching out to students who have fallen out of touch can be powerful.