ConcepTest CAT Arthur B. Ellis, Clark R. Landis, Kathleen Meeker Department of Chemistry University of Wisconsin-Madison WHY USE CONCEPTESTS? The instructor obtains immediate feedback on the level of class understanding. Students obtain immediate practice in using SMET terminology and concepts. Students have an opportunity to enhance teamwork and communication skills. Many instructors have reported substantial improvements in class attendance and attitude toward the course. WHAT ARE CONCEPTESTS? The instructor presents one or more questions during class involving key concepts, along with several possible answers. Students in the class indicate by, for example, a show of hands, which answer they think is correct. If most of the class has not identified the correct answer, students are given a short time in lecture to try to persuade their neighbor(s) that their answer is correct. The question is asked a second time by the instructor to gauge class mastery. Many variations on this general CAT exist. A video clip illustrating the method is part of this CAT. WHAT IS INVOLVED? Purpose of Assessment: To estimate class learning in real time. Instructor Preparation Time: Some time is needed to create ConcepTests. For some disciplines, hundreds of sample questions exist on websites as a time-saving resource. Preparing Your Students: Students require minimal training though sustained use in class helps students become comfortable with the method. Class Time: ConcepTests typically last from less than a minute to several minutes. Disciplines: Appropriate for all. Class Size: Best with classes of at least a dozen students. Successfully used in large lecture classes. Special Classroom/Technical Requirements: None. The method can be used in conjunction with worksheets, lecture demonstrations, and computer animations and filmclips. Individual or Group Involvement: Small group of 2 or 3 students. Analyzing Results: Minimal. Other Things to Consider: It is more difficult to predict how much material will be covered in a lecture. It may take a sustained effort for an instructor and class to become comfortable and work effectively with ConcepTests. Contents Description Assessment Purposes Limitations Learning Goals Instructor Goals Suggestions for Use Examples Variations Analysis Pros and Cons Theory and Research Links Sources Arthur B. Ellis, Clark R. Landis, Kathleen Meeker Description Many instructors have become far more satisfied with their SMET course simply by taking a few minutes during a typical lecture and posing a conceptual question called a ConcepTest to their students. Eric Mazur, a Harvard physics professor, developed this method for teaching undergraduate physics courses.1 These questions inform or calibrate students as to how well they understand what the instructor has identified as key ideas; and they calibrate instructors as to class mastery of these ideas at the time of their presentation. ConcepTests thus make the lecture a vehicle for bringing student and instructor expectations into alignment. ConcepTests are distinguished by a classwide vote-discuss-vote cycle built around a question that captures a key concept identified by the instructor. The format of a ConcepTest is shown schematically below, and illustrated in a quick-time video clip on the website (http://www.wcer.wisc.edu/archive/cl1). The ConcepTest process. An example of a ConcepTest that could be used in a variety of SMET courses is the following: If a blue solution cuts the amount of red laser light reaching a solar cell, which counts photons as photocurrent, in halfÉ What happens when a second identical solution is added? Ask your class to vote: "How many of you think that a quarter of the laser light will now reach the solar cell?" (Pause to observe response by, e.g., show of hands.) "How many of you think none of the light will now get to the solar cell?" (Wait as before.) If most of the class has correctly noted that one-quarter of the light will be transmitted, you can quickly affirm why this is correct and continue with the lesson. If the majority of the class does not select the correct answer, you ask your students to convince their neighbor(s) that their answer is correct. An option you might use is to give a hint, e.g., a reminder that each cuvette will transmit half of the light that enters it. After a short discussion period you have the class vote a second time. If the class has now selected the correct answer, a quick explanation is again appropriate; if the majority of the class has not reached the correct answer, a more detailed explanation can be presented. Sometimes the most effective ConcepTests are those for which the class has converged on an incorrect answer, as these questions identify a shared misunderstanding. Occasionally the class may even be thunderstruck by the correct answer, which, in this case, could be revealed by doing the experiment as a demonstration. In general, ConcepTests can be used to strip a concept to a particularly simple case for analysis, to extend a concept in a new direction, to couple concepts, and to confront misconceptions, which can be folded into the possible answers to a ConcepTest. ConcepTests are often constructed to occupy the class for short periods of time, ranging typically from less than half a minute to several minutes. Judging when best to interrupt the discussion is an important and acquired skill. In most classes the noise volume will slowly build as students come to task, and then the volume will drop. The students' attention should be regained before the volume increases again; when this occurs, students have often moved on to other subjects. With this non-graded, non-threatening format there is an opportunity for all students to discuss a concept with their classmates and to inform the instructor of the extent to which the concept is understood by the entire class. Because everyone is encouraged to participate, students are less fearful that their possibly incorrect answer will be publicly exposed to the entire class and thus are more likely to join in the discussion. Assessment purposes To obtain real-time feedback regarding student understanding. To determine the pace and direction of the remainder of the lecture. Limitations If the size of the class is too small (a dozen or fewer, e.g.), students may feel more conspicuous and self-conscious, hindering their participation. Instructors may need to encourage students to sit together to participate in ConcepTests if the lecture room has many more chairs than students; some instructors have grouped students into teams to work on ConcepTests during their lectures. Learning Goals Communicate verbally using SMET terminology Apply ideas and concepts Integrate concepts Work cooperatively with others Instructor Goals Obtain real-time feedback regarding student misconceptions and understanding Communicate the instructor's desire for students to be successful in the course Suggestions for Use Introduction of ConcepTests: ConcepTests can be used in virtually any kind of SMET course, from introductory courses for majors or nonmajors through upper level courses. If possible, instructors should start to introduce ConcepTests the first day of class and may wish to mention the use of this and/or other collaborative learning methods in their syllabus. Students are sometimes resistant to change and may resent the introduction of a new format later in the course. If ConcepTests are used throughout the course, this format will seem natural to the class and the instructor. While many classes embrace the method upon its first use, it is not uncommon for an instructor and class to need to work through several ConcepTests before both are comfortable with the method and the class learns that they are expected to participate. Presentation: It is straightforward to build ConcepTests into the planning of a lecture. They can be presented to the class verbally and/or with the questions and possible answers written on a blackboard or overhead transparency. Pre-prepared ConcepTests can also be presented using overhead transparencies or electronic PowerPoint presentations. The number of ConcepTests used during any one lecture can vary substantially. In a typical 50-minute lecture, instructors have used anywhere from one to about half-a-dozen ConcepTests. Demonstrations, designed to pique student interest and connect explanations with observable scientific phenomena, are traditionally presented with an introduction telling students what they will observe. A ConcepTest requiring students to predict the outcome is a convenient way to engage students and involve them in the demonstration. After discussion, the instructor does the demonstration and the students can determine whether or not their predictions were correct. ConcepTests can also be constructed around computer-based animations or film clips. ConcepTests may incorporate graphs or tables of data. These ConcepTests assess not only students' conceptual understanding, but also their ability to extract information. Markings on card used for responding to ConcepTest. Student response: Student responses to a ConcepTest can be evaluated in a number of ways: a raising of hands, a displaying of signs, a pushing of electronic touchpads, etc. Although a show of hands is the easiest and least expensive method, students may feel more comfortable and less conspicuous raising a sign whose orientation, color, and/or markings identifies their answer, as shown below. A similar method used in small classes has been to have students place a number of fingers on their chest which corresponds to the number of their chosen answer. Instructors may obtain a more accurate vote with the sign and finger methods since it is more difficult for a student to see how his or her classmates have voted. However, instructors should report the results of the vote to the class, so that students know the outcome. Instructors have the option of making voting mandatory or voluntary. In the latter case, signs and fingers mean all students vote at the same time, helping to identify students who have not yet voted. Some lecture rooms are equipped with touchpads. This technology makes it relatively easy to obtain statistical information on how students voted and how their votes were affected by discussion. Use of hands to respond to ConcepTest. Grading: Although ConcepTests are typically not themselves graded, instructors have found that a helpful element for the successful use of ConcepTests is an absolute grade scale, in which they guarantee at the very start of a course that a given level of course performance ensures a particular final grade (e.g., 85% and up guarantees an "A"; 70% a "B," etc.). This grading policy encourages students to help one another without fear of jeopardizing their grade, and it also enables students to track their progress in real time. The grade boundaries can always be lowered if exam scores are lower than anticipated, but the contract with the class is that they cannot be raised. ConcepTests promote a classroom culture of cooperative learning that can make SMET courses more user-friendly. Their use can, for example, lead to more effective student participation in study groups. Many instructors have seen substantially enhanced student performance as a result of using ConcepTests with other cooperative learning methods.2 Examples ConcepTests are easily incorporated into lectures. An introduction to acids would be part of any introduction to chemistry. A traditional lecture or two might include the following ConcepTests, or the instructor might choose a selection from a menu of the following ConcepTests (correct answers are bold-faced): ¥ introduce the autoionization of water to make protons and hydroxide ions, Question: Consider the reaction for the autoionization of water, H2O <==> H+ + OH- Would a beaker of water contain more protons or more hydroxide ions? Answers: protons, hydroxide ions, equal amounts of both Question: Which is the correct Lewis dot structure for the hydroxide ion? Answers: ¥ characterize the autoionization reaction as requiring heat Question: What happens to the amount of dissociation when a beaker of water is heated? Answers: more, less, same amount Question: The autoionization constant of water at body temperature is 2.8 x 10-14. The concentration of protons in pure water at body temperature is 1.7 x 10-7. Is the concentration of hydroxide ions the same, greater, or less than the concentration of protons? Answers: same, greater, or less ¥ develop the concept of the pH scale as -log[H+], where [H+] represents the concentration of protons in units of molarity, M. Question: If [H+] = 10-3 M, what is the pH? Answers: -3, 3, 7 Demonstration: The pH of 100 milliliters of a solution of the strong acid HCl is measured with a digital pH meter and shown to be about 1. Question: Roughly how many milliliters of water need to be added to reach a pH of about 2? Answers: 10, 100, 1000 Demonstration: Monitor the pH with each of these quantities of water added in turn. ¥ demonstrate that the pH of neutral water is about 7; present a 10-1 M solution of the strong acid HCl and demonstrate that its pH is about 1, i.e., the value that would be calculated from its concentration Question: When the leads of a light bulb are immersed in pure water, will the light bulb light up? Answers: Yes, no Question: When the leads of a light bulb are immersed in 0.1 M HCl, will the light bulb light up? Answers: Yes, no ¥ write the reaction for dissociation of HCl as HCl <==> H+ + Cl-; sketch beakers with different relative concentrations of HCl, H+ and Cl- and ask which is the best representation of the distribution of species present in the 0.1 M HCl solution Question: Which of the following "molecular pictures" best represents a solution of HCl? Answers: A. B. Variations Discussion options. Once the ConcepTest question has been posed and students given time to think and vote, the subsequent discussion period forms the core of what is called "think-pair-share" methodology. This process encourages a student to articulate her thoughts on a question and to listen to the thoughts described by classmates. These activities are a critical part of understanding a concept as it allows students to take ownership of a concept on their terms. It is axiomatic that we truly understand an idea when we can explain it to someone else in a way that lets them understand it, too. "Report out" is an extension of this think-pair-share approach, whereby volunteers from several groups of students are asked to explain to the class why they chose the answer that they did. This is done before the correct answer is presented so that students can examine and understand the reasoning behind choosing the incorrect as well as correct answers. Of course, with this method students must be encouraged not to become embarrassed by choosing the incorrect answer. Usually, as the report out method is used more often, students become increasingly comfortable with the idea of sharing their logic with the rest of the class. If there is still fear that students will not be comfortable sharing their answers aloud, a randomly selected anonymous group can be asked to put their answer and logic on an overhead transparency. In this way, the instructor can request classwide feedback to the answer without having to identify the students who provided it. Analysis People have collected data quantitatively for analysis after lecture; qualitatively, instructors can adjust the lecture based on class response. Some additional issues for which preliminary data are encouraging are: Increased class attendance Lower class attrition rate Better class performance Larger enrollment in subsequent SMET courses and number of majors Enhanced performance in subsequent SMET courses Reduced differential impact, i.e., reduced performance differences among different population groups in the class - men, women, students of different racial and ethnic backgrounds. Pros and Cons Value: The ability to receive immediate feedback on a ConcepTest is valued by students as well as instructors. Commonly, students comment on the value of a ConcepTest as a check on their understanding of the material. Amount of Coverage: Some lecturers and students are concerned that less material may be covered. A more likely outcome is that using this method encourages the instructor to focus more on the "big picture" and results in enhanced student understanding. Sometimes this immediate feedback allows for the coverage of more material. Many times a ConcepTest is posed and the overwhelming majority of the class can identify the correct answer immediately without having to discuss the question. In the all-lecture format the instructor might have spent considerable time explaining the topic as part of the scripted lecture. Instructor Preparation: The first time an instructor develops lectures around ConcepTests it may well take longer to prepare. However, many ConcepTests have already been developed and are available as a time-saving resource through websites. Once an instructor becomes comfortable with the method, preparing for a lecture can frequently take less time and be less stressful. Instead of a carefully scripted presentation that needs to be delivered by the instructor to fill completely the available class time, the essence of the ConcepTest approach is that the class period is punctuated with discussion periods of variable length that depend on the rate of class understanding. While this introduces the uncertainty of just how far the instructor will move through the presentation, it is easy to plan the storyline into the next lecture and simply pick up at the point where the class ends. Instructors may find preparing and delivering classes in this manner less stressful, as an implicit part of the preparation and delivery is that he or she does not always have to be in control. Moreover, the discussion periods provide an extension of preparation time. While the class is involved in discussion, the instructor has a wonderful opportunity to reflect on the best way to introduce the next idea, perhaps informed by speaking with student groups during the discussion periods; or casually to make a final check on the details of an upcoming demonstration that will be presented. Benefits: ConcepTests provide a mechanism for initiating the students (and instructor) into peer instruction strategies. ConcepTests can also help students break out of old habits that may inhibit learning, such as relying on short term memorization. An almost universal observation of instructors who use ConcepTests is that students become more articulate in using the language of science. The constant opportunity to use new terms and concepts in discussions with peers provides useful practice at a critical stage, facilitating the construction of a particular concept. The students not only hear correct answers expressed in their own words but often may realize that there is more than one approach to thinking about a problem. Students often find themselves motivated to come to class to consult with their peers, developing facility with the ideas and language of SMET disciplines in the classroom and enhancing their self-confidence. A common observation of instructors is that there are fewer empty seats in lecture after they started using ConcepTests. However, interactive lectures with ConcepTests are not valued by all students. Of course, no single technique will please everyone but there appears to be a minority of students whose expectations for lecture do not match the interactive style. A small number of students expect and want a more passive lecture. This may be especially true for those who were highly successful in a traditional course and are wary of trying something different. Theory and Research To our knowledge there has been relatively little research conducted on ConcepTests.1,2,4 Often ConcepTests have been used with other changes in content and pedagogy, making it difficult to decouple the impact of ConcepTests from the other course modifications. We urge the community to conduct research on this educational method. Some significant issues relating to course performance, retention, and attitude are listed above under Analysis. Links ConcepTest Websites: A library of physics ConcepTest questions is available and can be accessed on the world wide web at http://galileo.harvard.edu. Links to astronomy and biology sites, under construction in connection with the Galileo project, can be found here. A primary source of ConcepTests for use in Chemistry classrooms is the ConcepTest Website located at http://www.chem.wisc.edu/~concept/ This website contains ConcepTests suitable for use at all levels of Chemistry and is indexed by subdiscipline and by topic. Contributions to the website are requested and can be made by sending email to concept@fozzie.chem.wisc.edu. NSF Curriculum Reform Projects: The following websites provide overviews of the New Traditions and ChemLinks Curriculum Reform Projects, which help to support and disseminate ConcepTests for Chemistry. http://newtraditions.chem.wisc.edu/ http://chemlinks.beloit.edu/ Sources ConcepTest Video Tape: The New Traditions Project has produced a videotape illustrating how ConcepTests can be used to promote active learning in large lectures. Four teachers at the University of Wisconsin - Madison are shown in actual classes, each using this innovation in a slightly different manner. In all cases the objective is to encourage the students to process information for themselves and not simply to record passively the instructor's lecture. Arthur Ellis and Lorena Tribe are shown teaching Chemistry 109 (General & Analytical Chemistry, a science/engineering majors course), Charles Casey is shown teaching Chemistry 343 (Introductory Organic Chemistry), and G. Earl Peace is shown teaching Chemistry 103 (General Chemistry, a nonmajors course). Although each of these classes enrolls more than 150 students, the techniques shown have also been found to be effective in much smaller classes. Copies of this videotape are available in limited quantity. If you would like to receive a complimentary copy, please contact G. Earl Peace, Jr., at gpeacejr@facstaff.wisc.edu, or by phone (608-262-8674) or fax (608-265-8094). Art Ellis After teaching chemistry courses to undergraduates for fifteen years and watching student evaluations monotonically decline, I began exploring ways to improve my teaching techniques. Reading Sheila Tobias' books and encountering a description of Eric Mazur's Peer Instruction method provided me with perspectives on negative student experiences in my large lecture course and suggested that ConcepTests might make the course more accessible and user-friendly.5,6 Large lecture sections can be dehumanizing for many students, and actively engaging students during the lecture can significantly improve that experience. The first time I tried the method was the first day of class in 1993. After describing a simple experiment, I sketched two possible graphs on the blackboard for the outcome of the experiment. I then asked my class of 250 to turn to their neighbor, introduce themselves, and persuade their neighbor that their answer was correct. There was a moment of stunned silence, like the class was thinking "You mean he's going to let us talk in class?" Then the class erupted into animated discussion. The intensity of engagement was absolutely exhilarating. I was instantly sold on the method, which I have subsequently found to be remarkably versatile and effective. ConcepTests strongly support other features of the course designed to make it more user-friendly, such as an absolute grade scale and promotion of study groups.7,8 In comparing notes with other instructors, not everyone has as much success on their first try with ConcepTests. Phil Sadler and Eric Mazur have described the experience of using ConcepTests as experimenting with a different but improved tennis grip, in that you may reasonably expect to his some balls into the net initially, but eventually you can be far more successful in your game.9 Clark Landis A few years ago I was preparing for a new semester of General Chemistry and found myself bored. Not bored with the idea of teaching General Chemistry so much as bored by the thought of standing in front of a large group apparently disinterested students and talking. "Why don't science classes have the interactive dynamism that I recalled from literature and philosophy classes?", I wondered. Finding no easy answers, I resolved to change the class so that class had less lecture and more dialogue. Aware that the large size (>150 students) of my class would pose some problems, undaunted I forged ahead and reserved 10 minutes per class for open discussion. Visions of Oprah Winfrey and Phil Donahue danced in my head.

Reality was cruel. The first time, I tried to prompt open discussion by asking a question and asking for anyone to respond. After a long awkward silence, one person, and no one else, spoke up. This pattern continued for several lectures until the complaints started rolling in. "Why are you wasting our time asking us questions that we don't want to answer?" was a common refrain. About the same time, I went to a colleague's class and saw ConcepTests in action. Students voted, they discussed, they ENGAGED. In my next class, I asked a question, supplied possible answers, and asked them to vote for the correct answer. More than a third of the students participated, a 2,000% increase over my previous approach! As I continued the pattern: - question, possible answers, vote, discuss with your neighbors, vote again, etc. - the student participation grew. Not by changing my intentions, but by adapting my methods, the classroom grew lively, interactive, and fun. Since that experience, my old styles have not returned, nor has the boredom. Katie Meeker As a graduate teaching assistant for courses taught both with and without innovations such as ConcepTests, I have observed that discussion sections in which ConcepTests have been used often lead more easily to true discussions, as opposed to question and answer periods or time spent simply re-lecturing. Asking the students to work on group problems leads to a raucous noise as they tackle the problems, while in some other courses, getting students to talk about science together can be considerably more difficult. Students who have experienced ConcepTests are often more vocal in other aspects of the course; more willing to ask questions in discussion section and more collaborative in the laboratory. These students often demonstrate a great deal of clarity of thought when answering problem set and exam questions. 1. Mazur, E. Peer Instruction: A User's Manual; Prentice Hall: Upper Saddle River, NJ, 1997. 2. Hake, R.R. American Journal of Physics, 1998, 66(1), 64-74. 3. See http://www.uct.ac.za/projects/cbe/mcqman/mcqappc.html#C1. 4. Landis, C.R., Ellis, A.B., Lisensky, G.C., Lorenz, J.K., Meeker, K., Wamser, C.C. Chemistry ConcepTests: A Pathway to Interactive Classrooms, Prentice Hall, Inc., in preparation. Preprints of this booklet are available; email concept@chem.wisc.edu for more information. 5. Tobias, S. They're Not Dumb, They're Different: Stalking the Second Tier, Research Corporation: Tucson, AZ, 1990.Tobias, S. Revitalizing Undergraduate Science: Why Some Things Work and Most Don't, Research Corporation: Tucson, AZ, 1992. 6. Tobias, S. Revitalizing Undergraduate Science: Why Some Things Work and Most Don't, Research Corporation: Tucson, AZ, 1992. 7. Ellis, A.B. Chemtech, March 1995, 15-21. 8. Ellis, A.B. J. Chem. Educ., 1997,74(9), 1033-1040. 9. Mazur, E. In "Indicators of Success in Postsecondary SMET Education: Shapes of the Future; Synthesis and Proceedings of the Third Annual NISE Forum"; Workshop Report No. 6; University of Wisconsin: Madison, WI, 1998.