Assessment within the Broader Context of Course Development: A More In-Depth Look at Assessment - The Holistic ViewAssessment Drives Course Development
Assessment plays a dual role: It drives student learning and it provides important feedback for both students and instructors. In An Introduction to Assessment: The Basics, we focused primarily on the former of these two roles. Students want to "do well" (receive high grades) in their courses, and they use our assessment techniques as the means for determining what it is we expect them to learn and for diagnosing how well they are learning it. Students will adapt their mastery of course content to what our assessment techniques require of them. Thus, if we want students to achieve our course goals, we must choose appropriate assessment techniques that guide our students to those goals.
In this document, we build upon the feedback role of assessment. In particular, we discuss the feedback that assessment provides the instructor, and the ways in which assessment drives course development. We want our students to "do well" (attain our course goals), and we use our assessment techniques as the means for diagnosing how well our students are doing. We can modify our curriculum and instructional methods based on what our assessment techniques tell us about what students are learning compared to what we want them to learn. We must, therefore, choose assessment techniques that will give us useful feedback for refining our curriculum and our instructional methods so that we achieve our course goals.
What follows is a generalized model for course development, within which assessment plays a critical role. Not surprisingly, this course development model centers on course goals. As we will see, our goals determine the curriculum, instruction, and assessments that are best suited for the course, and we will see that assessment serves as the "feedback loop" wherein we evaluate the extent to which our curriculum and instruction choices are leading to the attainment of course goals so that we may then modify the curriculum and instructional methods based on this evaluation.
Curriculum, Instruction, and Assessment
While formalizing goals is the most important part of course development, it is only the first step. The path through the course development process can be envisioned as a "road map", with goals at the beginning, pointing the way, to assessment at the end, telling us if we have reached our destination or if we need to retrace our steps. This course development "road map" (Figure 1) provides a detailed set of directions, with specific actions to be taken at several signposts along the way. Starting from formalizing course goals, the "directions" are as follows:
Let's consider these steps in turn.
Translating Course Goals Into Measurable Student Outcomes
Thus, once goals have been formalized, the next step in course development is to translate the often abstract language of course goals into a set of concrete measurable student outcomes. Measurable student outcomes are specific, demonstrable characteristics--knowledge, skills, values, attitudes, interests--that will allow us to evaluate the extent to which course goals have been met. For each course goal, identify the principal outcomes one would expect from a student who has achieved that goal, keeping in mind that our ability to measure the achievement of course goals will be determined entirely on the basis of these measurable student outcomes. Of course, knowing what kinds of outcomes are actually measurable requires knowledge of the kinds of assessment techniques that are available, and what each technique can and cannot measure. Don't worry: we'll discuss different assessment techniques later, and the section on Classroom Assessment Techniques presents a variety of these assessment methods and how they connect with different measurable student outcomes. Figure 2 gives an example of translating a specific course goal (in the context of dental health) into measurable student outcomes.
Determining Levels of Expertise Required to Achieve Measurable Student Outcomes
What do we mean by "levels of expertise"? The various student outcomes that we assign to each course goal require different levels of mastery of course content. Some student outcomes require no more than students simply knowing the correct answer. However, many student outcomes require more sophisticated levels of understanding--or levels of expertise. Consider again the dental hygiene example above (Figure 2): The measurable student outcome of "knows the active ingredient in toothpaste" requires only that students memorize the correct answer (fluoride), while the outcome of "can describe how poor dental hygiene can lead to poor overall health" requires a much more sophisticated level of understanding, involving synthesis of multiple facts and concepts. Again, measurable student outcomes vary in the levels of expertise required to achieve them. Accordingly, the criteria by which we measure student success in achieving desired measurable student outcomes--our classroom assessment techniques--should be capable of assessing a variety of levels of expertise. In general, this means using a variety of classroom assessment techniques. Let's consider how to go about determining levels of expertise for our measurable student outcomes.
One of the most widely used ways of organizing levels of expertise is according to Bloom's Taxonomy of Educational Objectives. (Bloom et al., 1994; Gronlund, 1991; Krathwohl et al., 1956.) Bloom's Taxonomy (Tables 1-3) uses a multi-tiered scale to express the level of expertise required to achieve each measurable student outcome. Organizing measurable student outcomes in this way will allow us to select appropriate classroom assessment techniques for the course.
There are three taxonomies. Which of the three to use for a given measurable student outcome depends upon the original goal to which the measurable student outcome is connected. There are knowledge-based goals, skills-based goals, and affective goals (affective: values, attitudes, and interests); accordingly, there is a taxonomy for each. Within each taxonomy, levels of expertise are listed in order of increasing complexity. Measurable student outcomes that require the higher levels of expertise will require more sophisticated classroom assessment techniques.
The course goal in Figure 2--"student understands proper dental hygiene"--is an example of a knowledge-based goal. It is knowledge-based because it requires that the student learn certain facts and concepts. An example of a skills-based goal for this course might be "student flosses teeth properly." This is a skills-based goal because it requires that the student learn how to do something. Finally, an affective goal for this course might be "student cares about proper oral hygiene." This is an affective goal because it requires that the student's values, attitudes, or interests be affected by the course.
Bloom's Taxonomy is a convenient way to describe the degree to which we want our students to understand and use concepts, to demonstrate particular skills, and to have their values, attitudes, and interests affected. It is critical that we determine the levels of student expertise that we are expecting our students to achieve because this will determine which classroom assessment techniques are most appropriate for the course. Though the most common form of classroom assessment used in introductory college courses--multiple choice tests--might be quite adequate for assessing knowledge and comprehension (levels 1 and 2, Table 1), this type of assessment often falls short when we want to assess our students knowledge at the higher levels of synthesis and evaluation (levels 5 and 6). (Bloom et al., 1994; Tobias & Raphael, 1997.) Multiple-choice tests also rarely provide information about achievement of skills-based goals. Similarly, traditional course evaluations, a technique commonly used for affective assessment, do not generally provide useful information about changes in student values, attitudes, and interests.
Thus, commonly used assessment techniques, while perhaps providing a means for assigning grades, often do not provide us (nor our students) with useful feedback for determining whether students are attaining our course goals. Usually, this is due to a combination of not having formalized goals to begin with, not having translated those goals into outcomes that are measurable, and not using assessment techniques capable of measuring expected student outcomes given the levels of expertise required to achieve them. Using the CIA model of course development, we can ensure that our curriculum, instructional methods, and classroom assessment techniques are properly aligned with course goals.
Note that Bloom's Taxonomy need not be applied exclusively after course goals have been defined. Indeed, Bloom's Taxonomy and the words associated with its different categories can help in the goals-defining process itself. Thus, Bloom's Taxonomy can be used in an iterative fashion to first state and then refine course goals. Bloom's Taxonomy can finally be used to identify which classroom assessment techniques are most appropriate for measuring these goals.
Selecting Course Curriculum (Content) and Classroom Assessment Techniques
This is also the point in the course development "road map" where specific classroom assessment techniques (CATs) must be selected based upon measurable student outcomes and their associated levels of expertise. The FLAG site was designed primarily to help you through this step in the course development process. Provided for your use are a variety of CATs that have been tested in the field and which are authored by national experts in the use of that particular technique. To be sure, the CATs provided in the FLAG site are but a subset of innovative CATs available from a variety of resources. The CATs provided in the FLAG site should give you a good start.
The CATs selected in this step will provide the feedback you need to evaluate the extent to which yours course goals have been achieved. Thus, it is imperative that the CATs you select be properly matched with your measurable student outcomes. The FLAG site provides a facility for helping you do this.
Choosing and Implementing Instructional Methods
As with choosing curriculum and assessment, the choice of instructional methods must be guided by our course goals and, perhaps even more so, by the levels of expertise associated with measurable student outcomes. For example, suppose that two of the course goals in an introductory engineering course are that (1) students learn how to "design simple machines that satisfy realistic constraints" and (2) that students "can work effectively as part of a design team". A common measurable student outcome for these goals might be that "students, working in a team, can design a device, using simple raw materials, that protects an egg when dropped from a height of two stories". This measurable student outcome is at the "Organization" level of expertise (Bloom's Taxonomy of Educational Objectives for Skills-Based Goals; Table 2) because it requires students to "create new tasks or objectives incorporating learned ones". Traditional lecturing alone would not be a sufficient instructional method in this case. Instead, an instructional method that emulates teamwork and that promotes creative thought would be more appropriate. That is, a more collaborative instructional method is called for.
A variety of instructional methods have been developed for guiding students to the different levels of expertise represented by the goals of our course. One commonly used instructional method--collaborative learning--is described in detail in the College Level One's Collaborative Learning website. In fact, you will find that collaborative learning instructional methods are appropriate and useful for a wide variety of goals, outcomes, and levels of expertise.
Conducting Assessment and Evaluating Attainment of Goals: Closing the Feedback Loop
How, specifically, do we perform this evaluation? By what criteria do we know if we have achieved our goals? Our measurable student outcomes are the key: If these outcomes are realized, we will know that we have attained our course goals. Look at the assessment data. Did your students achieve the hoped-for outcomes, and at the desired levels of expertise? Using the engineering example from above, perhaps the egg survived but only because one student in the team did all of the work, i.e., the outcome related to teamwork was not realized. How might you modify the course to better foster effective teamwork? Perhaps students needed more guidance on how to work collaboratively; consider how the curriculum and/or instructional methods might be changed to accomplish this. Or perhaps it is the teamwork goal itself that needs refining. It is this important, evaluative step that allows you to determine the extent to which you are reaching your course goals and to decide if there are changes you would like to make.
We close this section by noting that the terms assessment and evaluation are often incorrectly used interchangeably. Assessment is the collecting of data to inform both the instructor and the student as to how the course is progressing (formative) or how it has ended (summative). Assessment involves gathering data via one or more classroom assessment techniques. Evaluation is what we do with these data once we have them. Once we have acquired the assessment data, it is up to us to judge the efficacy of our instructional methods, the content of our course, and the achievement of our course goals.
Assessment Is Feedback for both Students and Instructors
For our students, classroom assessment answers a different set of questions:
Answering these questions and others can inform and improve the quality of student learning in our classes.
We can not emphasize enough how important it is to actually write down your course goals and share them with your students. Our goals are what bind the course together (Figure 3). Our choices of curriculum, instruction, and assessment are all guided by--and held together by--our goals. Once your course goals are set, questions about instruction, assessment, and grading will be much more focused. This is a small step beyond the assessment strategies that most faculty are already doing; yet with a small investment in planning, the data acquired can provide valuable feedback for improving the quality of student learning. And ultimately, our students are what course development is all about.
Tell me more about assessment:
An Introduction to Assessment - the Basics
Assessment within the Broader Context of Course Development
Gina Brissenden and Timothy Slater
View Entire Primer