This page provides recommendations for creating good, high-quality examination questions. We also share technical and policy guidance for instructors who wish to modify their F2F exam processes to accommodate remote settings in Fall 2020.
Content
This page includes guidance on the following topics:
- Resources for creating quality exams
- How final exams may be altered
- Academic integrity considerations
- Preparing students for remote delivery exams
- Reminders about student conduct during remote delivery exams
- Student disability accommodations
1. Resources for creating quality exams
Use exam writing tips for constructing high-quality exams.
The wording and placement of exam items matter in creating a high-quality exam. For a clear discussion of what to pay attention to, see the following resources:
- Multiple-Choice Item-Writing Rules (2-page guide)
- Making Multiple Choice Tests More Effective (PPT from testing expert, Linda Suskie)
- Tomorrow’s Professor newsletter #1743
- Vanderbilt University Center for Teaching’s Multiple Choice Test Question Guide
Use multiple-choice questions to address higher-order thinking.
Writing items that address more than memorized terminology will help you assess what students understand at deeper levels. For examples of what these items look like in a multiple-choice format, see the following resources:
- Examples of Higher-Order Multiple Choice Items and Basics of Writing Tests (Presentation from 2008 testing workshop—see last 4 slides for examples)
- Best Practices for High-Quality Exam Questions (video of workshop presented by testing expert, Hoi Suen, Penn State Professor Emeritus, 2018)
2. How final exams may be altered from your usual exams for F2F courses
Change the question format.
Question format refers to options such as multiple-choice, short answer, essay, problem-solving, or original coding. If you previously taught with multiple-choice exams for your residential F2F course, you may want to consider changing questions to another format to better ensure academic integrity. Communicate your choice to students, so they know what to expect and can choose appropriate study strategies.
Change the exam format.
Exam format refers to closed-book or open-book. If you previously administered closed-book exams for your residential F2F course, you may want to consider adopting an open-book format. Communicate your choice to students, so they know what to expect and can choose appropriate study strategies. Also, consider communicating how to study for open-book exams, as many students think that “open book” means that they don’t need to study.
Changing the difficulty level of the exam is not recommended.
Adding many more questions to an exam with the same amount of time that you would give in a F2F course or making exam questions tricky, in an attempt to catch cheaters, is not good practice. It is unfair to students who study. Use other strategies to dissuade cheating (see Academic Integrity Considerations below).
3. Academic integrity considerations
What instructors can assume regarding academic integrity.
Students will likely have access to textbooks and course materials during the exam, even if you ask them to not use them.
Students may have communication with other students in the class via GroupMe or other texting mechanisms.
Students will likely multitask during the examination and may end up using the internet to look up answers.
What instructors can’t assume regarding academic integrity.
You cannot assume that all students will have the same technology (webcams, microphones, etc.), or reliable high-speed internet access.
You cannot assume that students will abide by restrictions for the use of electronic devices. Some exam proctoring tools might be able to tell if a student is accessing other applications and websites from the same computer. Do not assume that the student isn’t also accessing another nearby computer, tablet, or phone.
You cannot assume that students won’t have previous copies of exams and won’t share the current exam to future students. Regardless of the technology used to lock down the browser, we cannot assume that students won’t have the ability to copy the exam in its entirety to share with others.
Design exams according to the assumptions that instructors can and can’t make.
Design exams, based on reasonable assumptions, in ways that best preserve the integrity of the exam, the exam experience for students, and exam results. Specifically, consider the following:
- Employ tips for writing quality exams (and see Resources for Creating Quality Exams above)
- Write items that test more than memorization—consider Bloom’s Taxonomy as a guide for thinking about questions that address various levels of learning
- Use a quiz item bank with many questions
- Shuffle answer options on different questions
- Shuffle questions to create several versions of the exam
- Consider the appropriate amount of time for the exam and set a time limit
- Set a specific date and time of day for the exam
- Decide whether to proctor and the manner in which you will proctor
- If using Canvas for exams, consult academic integrity suggestions when testing in Canvas (see this PowerPoint Presentation, slides 6-7, from Online Faculty Experts Panel), and choose which measures you want to employ, given your particular exam context
- Consider the guidance above for creating quality exams, especially that from the Schreyer Institute about making multiple-choice tests more effective and from the College of IST workshop on Best Practices for High-Quality Exam Questions
4. Preparing students for remote delivery exams
Share with students how best to prepare for remote delivery exams.
In all cases, students should study for exams, whether they are open-book or closed-book. However, the type of studying that they should do should match what you are going to ask of them on the exam.
Will your exam test recall of vocabulary and recognition of examples of concepts? If so, memorization techniques and the ability to categorize examples will be important. Will your exam test the application of concepts in new settings? If so, study techniques that test students’ ability to explain concepts and processes, make connections, and perhaps do some problem solving will be important. Talk with your students about what they should be able to do and suggest appropriate strategies that will prepare them for the exam so that they don’t waste valuable studying time.
Share resources to assist students in organizing for studying:
5. Reminders about student conduct during remote delivery exams
Prior to giving an exam, express expectations with regard to academic integrity.
Students need to know that academic integrity matters—to students, to faculty, and to the quality of the degree that students earn. Share that Penn State expects all students (in residence and at a distance) to abide by the principles of academic integrity and that the same procedures are in place for dealing with misconduct whether courses are held face to face or in a remote setting.
Consider sharing the Penn State Code of Conduct with students.
6. Student disability accommodations
Students with registered disability accommodations should receive accommodations for exams, regardless of whether the course is remote or F2F. If you have students who have made an accommodation request, please reach out to Educational Equity and Diversity Programs to discuss alternative arrangements if you have questions about how to offer appropriate accommodations.
Students can read about disability resources on the Remote Learning FAQ page.
This material has been adapted from Penn State College of Information Sciences and Technology Fall 2020.
