This is the optimum way to compile a multiple-choice test, according to psychology research

Let's start with a quick multiple-choice test about multiple-choice tests: when designing them, should you a) avoid using complex questions, b) have lots of potential answers for each question, c) all of the above or d) none of the above? The correct answer is (a), though as we'll see, this was not a very well-crafted multiple-choice question. 

The issue of how best to design multiple-choice questions is important since they have been popular in both education and business settings for many years now. This is due to them being quick to administer and easy to mark and grade. Furthermore, many students often report preferring them over other test formats.

As well as being a useful assessment tool, if they are well-designed they can also aid learning. This is because of the Testing Effect – the way that retrieving knowledge helps consolidate it in memory. 

Thankfully in a recent paper in Journal of Applied Research in Memory and Cognition Andrew Butler of Washington University in St. Louis has reviewed the parallel literatures on how best to design multiple-choice tests for learning and assessment, and from this he's recommended six evidence-based tips:  

• Avoid using complex questions or answer formats. When things are too complex, it leads to misunderstandings and students guessing the answer. By keeping the format simple, it makes it a more reliable assessment of what the students do or don't know. 
• Create questions that focus on specific parts of knowledge or thought processes that you want to assess. For example, questions could focus on retrieval of a specific fact, contrasting two concepts, or applying a theory to a new situation.
• Avoid using "None of the Above" and/or "All of the above" as potential answers. The main issue with "None of the Above" is that if it is the correct answer, then the test-taker has been exposed to numerous false answers. This therefore represents a missed opportunity to reinforce what the correct answer was. "All of the Above" may be helpful when it is the correct answer because, as Butler explains, "the test taker is only exposed to the correct information", but may do more harm than good if it is the incorrect answer because of exposure to false answers.
• Be economical with the number of answer options. The topic will have a bearing on the optimal number to offer, but three potential answers usually provides enough difficulty whilst also being time efficient. Excessive potential answers may be problematic if the main aim is to enhance learning, as it will "expose test takers to a lot of incorrect information, [which] is worrisome because they could potentially learn it".
• Ensure the answer choice is "moderately difficult". If it is too easy or too hard then little will be learnt. Butler states that "the ideal difficulty level is a bit higher than the midpoint between chance and perfect performance".
• Ensure you provide feedback on the correct answers. When giving feedback to students, Butler notes that "one important caveat is that the effectiveness of feedback depends upon students being motivated to process it and their motivation tends to decrease over time". Therefore, if feedback is delayed, it would be wise to take steps to ensure that students are "required or incentivized to process it".

This paper has useful implications for anyone designing multiple-choice tests. Butler notes that more research is still needed, however. For example, one area that shows promise is having students assign a confidence rating to their answer, as this helps measure partial knowledge. However, the exact format for doing this is still being studied.

Further reading

—Multiple-Choice Testing in Education: Are the Best Practices for Assessment Also Good for Learning?

About the author

Bradley Busch (@Inner_Drive) is a registered psychologist and director of InnerDrive. He has worked with Premiership and International footballers and is the author of Release Your InnerDrive