Why do Assessment?
Are you asking too little of your class?
There is considerable evidence showing that assessment drives student learning. More than anything else, our assessment tools tell students what we consider to be important. They will learn what we guide them to learn through our assessments.
This Primer is designed to welcome you to the world of classroom assessment. The College Level One (CL-1) Team assumes you are here because you are interested in better ways to assess student learning in your class and in helping your students become more reflective and effective learners.
One goal of this website is to provide resources that enable you to begin this journey to more effective assessment of student learning. The first step is to articulate your course goals. Once you have identified your course goals, this website presents Classroom Assessment Techniques or CATs that are aligned with them. The CATs are a rich, eye-opening source of ideas and associated tools that have been extensively field-tested by your colleagues across the spectrum of STEM disciplines and post-secondary institutions. We encourage you to explore the CATs to see how you can work with your class to assess their understanding, skills, and attitudes through concept maps, conceptests, Fermi tests, interviews, portfolios, and other related techniques and tools. If you are not familiar with these forms of assessment, we are confident that you will discover, as we have, that they provide paths to a broader and deeper understanding of student learning for both you and your students.
We close by noting that assessment is undergoing exciting changes in college STEM courses. The overarching intent of this website is to capture the vitality of assessment. The CL-1 Team views assessment as a moving target and this website as a living product, providing both a mechanism for rapid dissemination of assessment-related developments and a forum for their discussion. In this spirit, the CL-1 Team invites you to join the growing number of college STEM instructors who are identifying and developing new tools that can be used to assess student learning and to share your ideas and experiences with us. We welcome your feedback and encourage you to contact us.
Why do it? Why do it in a particular way?
Why do it?
Why do it in a particular way?
Assessment is more than grades
Assessment is Feedback for Both Instructors and Students
Thus, in addition to providing us with valuable information about our students' learning, assessment should assist our students in diagnosing their own learning. That is, assessment should help students "become more effective, self-assessing, self-directed learners." (Angelo & Cross, 1993, p.4), Various classroom assessment techniques (CATs) have been developed with this in mind. The CATs provided in the FLAG site have been field-tested and shown to be effective at both measuring student mastery of content and at providing students with the feedback they need to become active participants in the learning process. Indeed, such feedback can positively influence what our students learn because assessment drives student learning.
Assessment Drives Student Learning
Given that this is the type of assessment our students most frequently encounter, and that it will eventually lead to their final course grades, students learn to study the content in our courses in an expeditious way that allows them to succeed in passing many first-year STEM courses without necessarily developing deep understanding of concepts. It is our assessment that drives students learning.
In fact, assessment drives student learning whether we want it to or not. The consequences of relying upon our "tried and true" assessment methods are profound; these assessment methods may actively promote superficial learning. If we wish to actively steer what our students learn, and how well they learn it, we must (1) actually decide what we want our students to take away from the course, and (2) choose our classroom assessment techniques appropriately (Anderson & Sosniak, 1994; National Research Council, 1996; Tobias & Raphael, 1997; Wiggins, 1998). The importance of setting course goals--articulating them and writing them down--cannot be overstated. Evaluating the extent to which we have attained our stated course goals is the primary motivation for why we "do assessment". Furthermore, ensuring that our assessment techniques can measure our stated goals is the reason for why we "do assessment in a particular way".
To evaluate attainment of course goals
Formalizing our goals is only the first step, however. We must also measure the extent to which we are attaining these goals. This is why we do assessment. Logically, we must choose classroom assessment techniques that are appropriately suited to measuring our particular goals. That is, we must align our assessment techniques with our stated goals.
To align assessment with stated goals
Furthermore, misaligned assessment techniques convey to our students the wrong message about what we want them to take from the course. As suggested previously, our choice of assessment technique drives student learning. (Anderson & Sosniak, 1994; National Research Council, 1996; Tobias & Raphael, 1997; Wiggins, 1998.)
But you may also wish to go beyond the basics of assessment. Concerns about assessment are not the only ones faced in the development and refinement of STEM courses; decisions about curriculum and instructional methods are equally important, and assessment plays a vital role in guiding these decisions. A more in-depth discussion of how curriculum, instructional methods, and assessment fit together is provided in "Assessment Within the Broader Context of Course Development," where we describe a generalized model for course development that builds upon the precepts that assessment drives student learning and that assessment provides feedback for both instructors and students. You will then find the FLAG site's facility for formalizing course goals and its suite of field-tested classroom assessment techniques to be of even greater value.
Assessment Drives Course Development
Assessment plays a dual role: It drives student learning and it provides important feedback for both students and instructors. In An Introduction to Assessment: The Basics, we focused primarily on the former of these two roles. Students want to "do well" (receive high grades) in their courses, and they use our assessment techniques as the means for determining what it is we expect them to learn and for diagnosing how well they are learning it. Students will adapt their mastery of course content to what our assessment techniques require of them. Thus, if we want students to achieve our course goals, we must choose appropriate assessment techniques that guide our students to those goals.
In this document, we build upon the feedback role of assessment. In particular, we discuss the feedback that assessment provides the instructor, and the ways in which assessment drives course development. We want our students to "do well" (attain our course goals), and we use our assessment techniques as the means for diagnosing how well our students are doing. We can modify our curriculum and instructional methods based on what our assessment techniques tell us about what students are learning compared to what we want them to learn. We must, therefore, choose assessment techniques that will give us useful feedback for refining our curriculum and our instructional methods so that we achieve our course goals.
What follows is a generalized model for course development, within which assessment plays a critical role. Not surprisingly, this course development model centers on course goals. As we will see, our goals determine the curriculum, instruction, and assessments that are best suited for the course, and we will see that assessment serves as the "feedback loop" wherein we evaluate the extent to which our curriculum and instruction choices are leading to the attainment of course goals so that we may then modify the curriculum and instructional methods based on this evaluation.
Curriculum, Instruction, and Assessment
While formalizing goals is the most important part of course development, it is only the first step. The path through the course development process can be envisioned as a "road map", with goals at the beginning, pointing the way, to assessment at the end, telling us if we have reached our destination or if we need to retrace our steps. This course development "road map" (Figure 1) provides a detailed set of directions, with specific actions to be taken at several signposts along the way. Starting from formalizing course goals, the "directions" are as follows:
Let's consider these steps in turn.
Translating Course Goals Into Measurable Student Outcomes
Thus, once goals have been formalized, the next step in course development is to translate the often abstract language of course goals into a set of concrete measurable student outcomes. Measurable student outcomes are specific, demonstrable characteristics--knowledge, skills, values, attitudes, interests--that will allow us to evaluate the extent to which course goals have been met. For each course goal, identify the principal outcomes one would expect from a student who has achieved that goal, keeping in mind that our ability to measure the achievement of course goals will be determined entirely on the basis of these measurable student outcomes. Of course, knowing what kinds of outcomes are actually measurable requires knowledge of the kinds of assessment techniques that are available, and what each technique can and cannot measure. Don't worry: we'll discuss different assessment techniques later, and the section on Classroom Assessment Techniques presents a variety of these assessment methods and how they connect with different measurable student outcomes. Figure 2 gives an example of translating a specific course goal (in the context of dental health) into measurable student outcomes.
Determining Levels of Expertise Required to Achieve Measurable Student Outcomes
What do we mean by "levels of expertise"? The various student outcomes that we assign to each course goal require different levels of mastery of course content. Some student outcomes require no more than students simply knowing the correct answer. However, many student outcomes require more sophisticated levels of understanding--or levels of expertise. Consider again the dental hygiene example above (Figure 2): The measurable student outcome of "knows the active ingredient in toothpaste" requires only that students memorize the correct answer (fluoride), while the outcome of "can describe how poor dental hygiene can lead to poor overall health" requires a much more sophisticated level of understanding, involving synthesis of multiple facts and concepts. Again, measurable student outcomes vary in the levels of expertise required to achieve them. Accordingly, the criteria by which we measure student success in achieving desired measurable student outcomes--our classroom assessment techniques--should be capable of assessing a variety of levels of expertise. In general, this means using a variety of classroom assessment techniques. Let's consider how to go about determining levels of expertise for our measurable student outcomes.
One of the most widely used ways of organizing levels of expertise is according to Bloom's Taxonomy of Educational Objectives. (Bloom et al., 1994; Gronlund, 1991; Krathwohl et al., 1956.) Bloom's Taxonomy (Tables 1-3) uses a multi-tiered scale to express the level of expertise required to achieve each measurable student outcome. Organizing measurable student outcomes in this way will allow us to select appropriate classroom assessment techniques for the course.
There are three taxonomies. Which of the three to use for a given measurable student outcome depends upon the original goal to which the measurable student outcome is connected. There are knowledge-based goals, skills-based goals, and affective goals (affective: values, attitudes, and interests); accordingly, there is a taxonomy for each. Within each taxonomy, levels of expertise are listed in order of increasing complexity. Measurable student outcomes that require the higher levels of expertise will require more sophisticated classroom assessment techniques.
The course goal in Figure 2--"student understands proper dental hygiene"--is an example of a knowledge-based goal. It is knowledge-based because it requires that the student learn certain facts and concepts. An example of a skills-based goal for this course might be "student flosses teeth properly." This is a skills-based goal because it requires that the student learn how to do something. Finally, an affective goal for this course might be "student cares about proper oral hygiene." This is an affective goal because it requires that the student's values, attitudes, or interests be affected by the course.
Bloom's Taxonomy is a convenient way to describe the degree to which we want our students to understand and use concepts, to demonstrate particular skills, and to have their values, attitudes, and interests affected. It is critical that we determine the levels of student expertise that we are expecting our students to achieve because this will determine which classroom assessment techniques are most appropriate for the course. Though the most common form of classroom assessment used in introductory college courses--multiple choice tests--might be quite adequate for assessing knowledge and comprehension (levels 1 and 2, Table 1), this type of assessment often falls short when we want to assess our students knowledge at the higher levels of synthesis and evaluation (levels 5 and 6). (Bloom et al., 1994; Tobias & Raphael, 1997.) Multiple-choice tests also rarely provide information about achievement of skills-based goals. Similarly, traditional course evaluations, a technique commonly used for affective assessment, do not generally provide useful information about changes in student values, attitudes, and interests.
Thus, commonly used assessment techniques, while perhaps providing a means for assigning grades, often do not provide us (nor our students) with useful feedback for determining whether students are attaining our course goals. Usually, this is due to a combination of not having formalized goals to begin with, not having translated those goals into outcomes that are measurable, and not using assessment techniques capable of measuring expected student outcomes given the levels of expertise required to achieve them. Using the CIA model of course development, we can ensure that our curriculum, instructional methods, and classroom assessment techniques are properly aligned with course goals.
Note that Bloom's Taxonomy need not be applied exclusively after course goals have been defined. Indeed, Bloom's Taxonomy and the words associated with its different categories can help in the goals-defining process itself. Thus, Bloom's Taxonomy can be used in an iterative fashion to first state and then refine course goals. Bloom's Taxonomy can finally be used to identify which classroom assessment techniques are most appropriate for measuring these goals.
Selecting Course Curriculum (Content) and Classroom Assessment Techniques
This is also the point in the course development "road map" where specific classroom assessment techniques (CATs) must be selected based upon measurable student outcomes and their associated levels of expertise. The FLAG site was designed primarily to help you through this step in the course development process. Provided for your use are a variety of CATs that have been tested in the field and which are authored by national experts in the use of that particular technique. To be sure, the CATs provided in the FLAG site are but a subset of innovative CATs available from a variety of resources. The CATs provided in the FLAG site should give you a good start.
The CATs selected in this step will provide the feedback you need to evaluate the extent to which yours course goals have been achieved. Thus, it is imperative that the CATs you select be properly matched with your measurable student outcomes. The FLAG site provides a facility for helping you do this.
Choosing and Implementing Instructional Methods
As with choosing curriculum and assessment, the choice of instructional methods must be guided by our course goals and, perhaps even more so, by the levels of expertise associated with measurable student outcomes. For example, suppose that two of the course goals in an introductory engineering course are that (1) students learn how to "design simple machines that satisfy realistic constraints" and (2) that students "can work effectively as part of a design team". A common measurable student outcome for these goals might be that "students, working in a team, can design a device, using simple raw materials, that protects an egg when dropped from a height of two stories". This measurable student outcome is at the "Organization" level of expertise (Bloom's Taxonomy of Educational Objectives for Skills-Based Goals; Table 2) because it requires students to "create new tasks or objectives incorporating learned ones". Traditional lecturing alone would not be a sufficient instructional method in this case. Instead, an instructional method that emulates teamwork and that promotes creative thought would be more appropriate. That is, a more collaborative instructional method is called for.
A variety of instructional methods have been developed for guiding students to the different levels of expertise represented by the goals of our course. One commonly used instructional method--collaborative learning--is described in detail in the College Level One's Collaborative Learning website. In fact, you will find that collaborative learning instructional methods are appropriate and useful for a wide variety of goals, outcomes, and levels of expertise.
Conducting Assessment and Evaluating Attainment of Goals: Closing the Feedback Loop
How, specifically, do we perform this evaluation? By what criteria do we know if we have achieved our goals? Our measurable student outcomes are the key: If these outcomes are realized, we will know that we have attained our course goals. Look at the assessment data. Did your students achieve the hoped-for outcomes, and at the desired levels of expertise? Using the engineering example from above, perhaps the egg survived but only because one student in the team did all of the work, i.e., the outcome related to teamwork was not realized. How might you modify the course to better foster effective teamwork? Perhaps students needed more guidance on how to work collaboratively; consider how the curriculum and/or instructional methods might be changed to accomplish this. Or perhaps it is the teamwork goal itself that needs refining. It is this important, evaluative step that allows you to determine the extent to which you are reaching your course goals and to decide if there are changes you would like to make.
We close this section by noting that the terms assessment and evaluation are often incorrectly used interchangeably. Assessment is the collecting of data to inform both the instructor and the student as to how the course is progressing (formative) or how it has ended (summative). Assessment involves gathering data via one or more classroom assessment techniques. Evaluation is what we do with these data once we have them. Once we have acquired the assessment data, it is up to us to judge the efficacy of our instructional methods, the content of our course, and the achievement of our course goals.
Assessment Is Feedback for both Students and Instructors
For our students, classroom assessment answers a different set of questions:
Answering these questions and others can inform and improve the quality of student learning in our classes.
We can not emphasize enough how important it is to actually write down your course goals and share them with your students. Our goals are what bind the course together (Figure 3). Our choices of curriculum, instruction, and assessment are all guided by--and held together by--our goals. Once your course goals are set, questions about instruction, assessment, and grading will be much more focused. This is a small step beyond the assessment strategies that most faculty are already doing; yet with a small investment in planning, the data acquired can provide valuable feedback for improving the quality of student learning. And ultimately, our students are what course development is all about.