How can faculty design their online courses?
Page 3: Assessments
Information on this page was adapted from content published by Vanderbilt University’s Center for Teaching (CFT), including:
- Assessing Student Learning, Bandy, J.
- Developing Online Assessments of Student Learning in a Hurry, Johnson, S.M.
Now that you’ve created goals and objectives for your course, it’s time to tackle the second part of backward design: thinking about the assessments you will use to determine the extent to which students have mastered those goals. Measuring student performance can be accomplished through the use of both formal and informal assessments, including homework, in-class assignments, exams, or projects. Whether formal or informal, effective assessments can also:
- Inform the ways in which you will approach and deliver instruction
- Help you plan instruction that more efficiently and effectively targets specific student needs, subsequently improving learning outcomes
- Help students know whether or not they are on track
In this interview, Joe Bandy provides some considerations for selecting assessment tools.
Joe Bandy, PhD
Assistant Director, Center for Teaching
Vanderbilt University
Joe Bandy, PhD
Considerations and cautions for online quizzes and tests
Well, I would start by learning the course management system or learning management system that your institution is using since that is likely to be the place that’s going to have the most functionality. LMSs—learning management systems—are quite elaborate. They have a lot of functions that go beyond merely using them as a repository for readings or syllabi. If you dig into them, or you undergo more advanced training in them, you can see that they are often capable of quizzes, surveys, discussion boards, blogs, rubrics, grading, grade books and other features that will aid in the assessment models that any instructor would like to use. Within that, I would say that discussion boards are particularly useful for having students respond to prompts about readings and central course questions, as well as having them share thoughts in the form of writing or video or audio that can build learning community with one another. They can afford both summative and formative types of assignments that is, graded or ungraded assignments, allowing for all kinds of interactive and collaborative learning with students.
One more idea is for courses that are focused on writing. Writing assignments can be handed in as texts in course management systems. And those can be there for the faculty or they can be shared in the form of discussion posts, which is great for peer review. There are a set of tools for peer review that are really useful. So Peerceptiv is one tool that instructors may like. It’s a platform through which students can submit papers and automatically be assigned peers who will review those papers and even assess how much their peers are engaging with the papers. And given the kinds of comments that their peers get, those are then produced in a report. It goes to the instructor and the instructor can use it as they will to assess or evaluate how students’ grade should be.
Joe Bandy, PhD
First steps
I will also say that there are many instructors who prefer quizzes and tests, and you can use the functions afforded by your course management systems to do that. Or you can use other types of platforms like Gradescope, which offers a grading platform for assignments like quizzes and problem sets, and is much more popular in the STEM fields. One word, however, about quizzes and tests online is that the closed book timed exam is challenging online since there are both connectivity issues for students as well as academic integrity issues with students learning at home where exams can’t be proxied in time and the way they can be face-to-face. Those both pose problems, and I see more and more faculty moving away from the closed book timed exam and towards more open book or untimed essays. That prevents them from having to engage with lockdown browsers or proxy mechanisms online that, while they do exist and sometimes work well, often encounter problems. So moving to an open book, untimed exam circumvents the needs for such things and often offers students forms of assessment that maximize learning opportunities.
It’s especially important when designing an online course that you align your assessment methods and tools with your learning goals. If you know what you want students to achieve (learning objectives), you’ll be able to determine whether they’ve done so (assessment). To begin this process, it’s helpful to know what purpose the assessment will serve. The table below outlines the three most common types of assessments, each with a different purpose, and offers examples that can be used either synchronously or asynchronously.
| Type/Purpose | Definition | Examples |
| Diagnostic assessment |
Often referred to as pre-assessment, it can be used by teachers before instruction to:
|
|
| Formative assessment |
This ongoing evaluation of student learning during instruction can be used to:
|
|
| Summative assessment |
Evaluation administered after instruction can be used to:
|
|
Developing a Comprehensive, Balanced Assessment Plan
Digging Deeper: Assessing Students
The resources below, from Vanderbilt University’s Center for Teaching, provide more in-depth considerations, suggestions, and tips for selecting and developing student assessments.
Developing a comprehensive, balanced assessment plan will help ensure that you are incorporating the three types of assessments discussed above in online and hybrid courses. As you create this plan you need to make certain that the assessments:
- Align with your course’s goals and learning objectives
- Provide information that can guide instruction
- Provide sufficient feedback to instructors and students to ensure that students are on track to meet learning goals
- Measure student learning to determine mastery of course objectives
Further, when you select assessments, you should give serious consideration to each of the following questions:
- Do the assessments observe and/or measure student action?
- In other words, each assessment should:
- Ask students to use their newly learned knowledge and skills (e.g., by applying them, evaluating them, or creating something new)
- Measure the extent to which students have achieved the learning outcomes
- Do the assessments align with one or more learning objectives?
- By aligning assessments with learning objectives, you ensure that:
- You are measuring the outcome (i.e., the knowledge and skills you have targeted)
- You are monitoring whether students are ready to move on to the next skill (i.e., scaffolding)
- Students are able track their progress in the course
- Does your plan include an appropriate mixture of assessment types?
- Your plan should include:
- Diagnostic, formative, and summative assessments
- Assignments as well as quizzes and tests
Questions adapted from University of California, Davis, module series.
Assessment Tools
Once you have created your assessment plan and know the types of assessments you are going to use, you are ready to identify assessment tools. You might want to begin by using tools that are incorporated in your learning management system (LMS), although you can also include external tools. Regardless, be strategic about the tools you use. It is best to start with one or two tools that you are familiar with or feel confident in your use of. The more tools you use, the more support you will need to provide to students—they may have a learning curve or need your support to use each new tool.
Assessment Tools
The Blended & Online Learning Design (BOLD) Fellows Program—a partnership between the Center for Teaching and Center for the Integration of Research, Teaching, and Learning at Vanderbilt University—provides an analysis of online tools that can be used for online learning activities and assessments. To view this information, visit the BOLD Fellows Tools page. Next, click on the “Tools” drop down menu and select the method of assessment (e.g., quizzing, portfolios) that you are interested in. From there you will find general information about the method, potential uses, pros and cons, as well as a variety of tools that can be used to implement it in your course.
For details about using the quiz or test functions in your specific LMS, choose one of the links below.
Tips
When you teach an online or a hybrid course, you should consider which assessments align best with your learning goals. Beyond that, you might want to take into account the following.
-
- Manage your time by automating when possible. Online grading can be a time-consuming process, but there are tools available to help mitigate these demands.
- Check your tools to see whether they allow you to pre-load feedback that you are likely to offer to multiple students.
- Consider providing audio and video feedback to students. This can feel much more personal and help strengthen your rapport with them. Source: Johnson, S., Developing Online Assessments of Student Learning in a Hurry.
- Maximize your return on investment. If you’ve already been using online tools in your course, consider:
- The tools you know well
- How you might leverage your existing knowledge
- Which tools will give you the most useful data on student performance
-
If you typically use a certain type of assessment in a face-to-face environment, but the tools required to do the same kind of assessment online are overly challenging, consider whether it is still a good investment to go ahead with your existing exam. If not, you might decide to switch (e.g., from a discrete-item exam to a reflective paper; from a performance assessment to an online quiz). Source: Johnson, S., Developing Online Assessments of Student Learning in a Hurry.
- Manage your time by automating when possible. Online grading can be a time-consuming process, but there are tools available to help mitigate these demands.
For Your Information
Although it might be time consuming to enter an existing test into your LMS, once you do so there are several advantages:
-
-
- Automatic scoring saves you time grading and offers students immediate feedback.
- Test scores can be automatically uploaded to your gradebook, again saving you time.
- Item analysis information helps you evaluate and improve the quality of your test questions.
- Test analytics that indicate often-missed questions can help you identify content areas you might wish to revisit. You might even consider this information the next time you teach the course.
-
- Seek assistance. Rest assured: You’re not the only one to struggle with online assessments. Connect with colleagues with similar challenges or similar kinds of courses to crowdsource solutions. If you have questions about the tools available in your LMS, check out on-demand resources or individual supports available at your college or university. Source: Johnson, S., Developing Online Assessments of Student Learning in a Hurry.
- Write test questions. Online exams should emphasize synthesis questions over recall questions whose answers are easy to obtain through Internet searches.
-
Digging Deeper: Assessing Students
- Tools for Grading – Sample Rubrics and Spreadsheets Vanderbilt University, Center for Teaching
Emphasize student-created work. When you assign grades in online courses, it’s recommended that you give more weight to student-created work than to online tests or exams. When you design and grade student-created work, rubrics can be a useful tool. Most online learning platforms enable instructors to create and share rubrics with students relatively easily.
- Maintain academic integrity for tests. There are a number of tools to help make your online test more secure and/or to prevent students from cheating. For example, your LMS might include a tool that detects possible instances of plagiarism. Additionally, most LMS multiple-choice quiz or test generators allow you to select the option of item and answer randomization. This generates a unique version of the test for each student.
For Your Information
- Faculty should assess student learning through a wide variety of formats so that students do not experience barriers due to their backgrounds, abilities or disabilities, or technology access. Faculty should also ensure that assessments are fair insofar as every student has what they need to be successful, including resources, appropriate standards, academic content, and support.
Example Course Development Plan: Assessments
Now that we’ve discussed selecting assessments and creating an assessment plan, let’s take a look at some examples.
Activity
Apply what you have learned on this page to create an assessment plan and to choose assessments for your own course.
- Draft an assessment plan for at least one course goal and its corresponding learning objectives. Continue using the template that you downloaded from Page 2 and on which you entered course goals and learning objectives.
- Make sure your plan considers the following factors:
- Do the assessments observe and/or measure student action?
- Do the assessments align with one or more learning objectives?
- Does your plan include an appropriate mixture of assessment types?
- Explore your LMS and select an assessment tool to use for one of your class assessments. Try it out on friends, colleagues, or teaching assistants. Did it work as you expected? Could you use a different tool to measure student learning?
