
Mathematical Thinking CATs  Fault Finding and Fixing  Plausible Estimation
Creating Measures  Convincing and Proving 
Reasoning from Evidence

Classroom Assessment Techniques
'Creating Measures' Tasks
(Entire CAT)


Malcolm Swan
Mathematics Education, University of Nottingham
Malcolm.Swan@nottingham.ac.uk
Jim Ridgway
School of Education,
University of Durham
Jim.Ridgway@durham.ac.uk
WHY USE CREATING MEASURES TASKS?
We constantly "mathematize," or construct measures for, physical and social phenomena and use these models to make decisions about our everyday lives. These can vary from measures of simple quantities (such as "speed" or "steepness") to complex and subjective social ones (such as "quality of life" or "best universities"). Since these measures are mathematical models of some phenomenon, they are open to criticism and improvement, especially when considering their usefulness.
These tasks (an example) provide a fun and interesting way to assess your students' abilities to "mathematize" concepts and show students that there can be many different formal, quantitative measures of such concepts. More importantly, they emphasize that measures differ in their utility; some are more useful than others in representing concepts.
WHAT ARE CREATING MEASURES TASKS?
A 'Creating Measures' task consists of a series of questions that prompt students to evaluate an existing measure of an intuitive concept, and then, to create and evaluate their measure of this concept (for example, to determine the steepness of a staircase). These tasks take about 45 minutes of class time, but can also be done outside of class.
WHAT IS INVOLVED?
Instructor Preparation Time: 
Minimal if use existing tasks.

Preparing Your Students: 
Students will need some coaching on their first task.

Class Time: 
45 minutes.

Disciplines: 
Appropriate for all, requires little mathematical technique.

Class Size: 
Any.

Special Classroom/Technical Requirements: 
None.

Individual or Group Involvement: 
Either.

Analyzing Results: 
Intensive for formal scoring for large classes. Best used as an informal way to get your students thinking mathematically.

Other Things to Consider: 
Fairly demanding task for students who are unfamiliar with openended problems.

Description
"Creating Measures" tasks consist of a series of questions that require students formally and quantitatively to define a measure of some concept. The concepts are ones of which students are intuitively aware, but which they have probably never attempted to describe mathematically. For example, students are asked to consider how they would define and quantify such concepts as "squareness," "sharpness," or "compactness." Usually, there are no formally correct and universally agreedupon answers to such questions, but some definitions are clearly more useful than others. Students are required to not only come up with a measure of the concept, but also to evaluate how well that measure works as a mathematical description of a concept.
To offer one example, let us consider the problem of finding a measure for the "squareness" of a rectangle. One candidate might be: "the difference between the longest and shortest sides." This may seem sensible at first glance as it gives a measure of zero for squares and a larger measure as the difference between the two dimensions of the rectangle increases. One problem is, however, that the measure depends on the units used for measurement. A second related problem is that two mathematically similar rectangles will give a different value for the measure. Clearly a measure that does not depend upon the dimensions of the object would be better, such as "longest side/shortest side."
Example of "Creating Measures" Task
"Steepness" of Staircases
This problem gives you the chance to
 criticize a given measure for the concept of 'steepness;'
 invent your own ways of measuring this concept; and
 examine the advantages and disadvantages of different methods.
Warmup
Without measuring anything, put the above staircases in order of steepness.
 Someone has suggested that a good measure of 'steepness' is to calculate the difference Height of step  length of step for each staircase. Use this definition to put the staircases in order of steepness. Show all your work.
 Using your results, give reasons why Height of step  length of step may not be a suitable measure for "steepness".
 Invent a better way of measuring "steepness". Describe your method carefully.
 Place the staircases in order of "steepness" using your method. Show all your work.
 Do you think your measure is a good way of measuring "steepness"? Explain your reasoning carefully.
 Describe a different way of measuring "steepness".
 Compare the two methods you invented. Which is best? Why?

Thus "creating a measure" in most cases involves combining collections of measurements in new ways and considering the dimensionality of the result.
Assessment Purposes
There are three assessment purposes:
Limitations
"Creating Measures" tasks are meant to measure the students' ability to "mathematize" a concept, or their ability to define a numerical measure of a concept, as well as evaluate the usefulness of that measure. This is important in that students can see that measures used to model phenomena in science are socially agreedupon notions that exist primarily because of their usefulness. For most tasks, little mathematical technique is required and so these tasks do not assess algebraic techniques.
Teaching Goals
Suggestions for Use
Introducing "Creating Measures" tasks for the first time
Many students will be unfamiliar with the openended nature of 'Creating Measures' tasks, and you may receive some initial resistance from your students about this. You can reduce student resistance in three ways.
 First, prepare your students by telling them that the goal of these tasks is to get them thinking mathematically; you are looking for their ability to define and evaluate a measure. You may need to "sell" these tasks to the students since many will not be accustomed to openended problems that don't assess manipulation of formulas. You may choose to emphasize the ubiquity of measures in all sciences, and in everyday life, with examples of welldefined measures (chosen from science e.g., heat and temperature) and less well defined measures ('happiness'; 'quality of teaching'). You might flag the importance of good measures for any kind of modeling in mathematics and science.
 Second, to help your students adjust to these types of problems, the first "Creating Measures" task should be a nongraded, inclass, groupbased exercise. For, if the task is nongraded, students can work on it without fear of "messing up their grade" and, if the task is done inclass, students can receive help from you as they work through the exercise. And, finally, if the task is groupbased, then the students can struggle together and receive support from one another.
 Finally, students will be anxious to know what a "good" answer is  you can provide them with various rubrics (see analysis section of this document) that describe the kinds of answers you expect to see and examples of each (or at least of a good answer).
Providing guidance as students work on 'Creating Measures' tasks
Clear guidance is provided for each task in the downloaded materials. The amount of guidance that students need should decrease as they become familiar with these types of problems. The amount and type of help you provide the students depends upon your goals for the task. For instance, if your primary goal is to assess how well students have learned to create measures in the course of teaching, you may choose to provide very little assistance.
Reporting out of individual or group work
If you decide to have the students congregate as a large group to discuss their solutions, it is again helpful to decide the degree to which you will participate in these discussions (which will depend upon your goals for the session). For instance, you can facilitate the students' discussion, having them defend their ideas and write their ideas on the board, while adding almost none of your own. Your comments might be designed to encourage students to critique their own, and others', measures in a more sophisticated way. Or, you might lead the discussion, soliciting student measures and offering a structured critique of the advantages and disadvantages of different measures. Critiques using characteristics of good and poor measures (e.g., insensitivity to scale; appropriate rank order with respect to 'intuitive' orderings; or, measures that are bounded) will reinforce the key ideas in the CAT.
Formal and informal use of "Creating Measures" tasks
These tasks can be used formally or informally. In formal assessment (where you grade the assignment as an examination), do not intervene except where specified. Even modest interventions  reinterpreting instructions, suggesting ways to begin, offering prompts when students appear to be stuck  have the potential to alter the task for the student significantly.
In informal assessment (an exercise, graded or nongraded), you may want to be less rigid in giving the students help. Under these circumstances, you may reasonably decide to do some coaching, talk with students as they work on the task, or pose questions when they seem to get stuck. In these instances you may be using the tasks for informal assessmentsobserving what strategies students favor, what kinds of questions they ask, what they seem to understand and what they are struggling with, and what kinds of prompts get them unstuck. This can be extremely useful information in helping you make ongoing instructional and assessment decisions. However, as students gain experiences with these kinds of tasks, the amount of coaching you do should decline and students should rely less on this kind of assistance.
Group work versus individual work
The openended nature of "Creating Measures" tasks makes for great group work problems. Students can discuss various measures and their merit and are likely to come up with many more ideas than if they worked alone. The CL1 Collaborative Learning web site can provide instructions on how to use group work effectively within the classroom. However, individual work may give you more clues as to each student's sophistication with this type of problem.
Presumed background knowledge
One nice attribute of "Creating Measures" tasks is that they require little mathematical technique. Students do need to have a basic knowledge of geometry concepts (area, perimeter, length), basic numeric skills (multiplication, addition, subtraction, division), use of formulas, and some algebra (the notion of a variable). The types of measures that students create will depend upon their mathematical knowledge, for the more sophisticated a student is with mathematical tools, the more likely the student will be able to come up with more sophisticated measures. However, the quality of a solution lies not in the complexity of the mathematical form, but in its functionality.
StepbyStep Instructions
 Prepare by reading through the "Creating Measures" task on your own and coming up with your own solutions.
 Hand out copies of the task to students, either working individually or in groups.
 State your goals for the 'Creating Measures' task, emphasizing that they should be able to defend both their assumptions and the reasoning that leads to their answer, and the desirable 'metric' functions of the measure.
 Walk around and listen to students as they discuss and work through the problems, providing guidance as necessary.
 Have students present their solutions, either in written or verbal form.
Variations
The tasks included in this site can be downloaded and used without modification. If you choose to develop your own 'Creating Measures' task, you can follow the pattern used in these tools:
 Give students a warmup discussion to make sure they understand the concept that they are trying to mathematize;
 Provide a measure, ask them to apply this measure to various examples, and to evaluate the usefulness of that measure;
 Ask them to come up with their own measure and apply it to various examples; and,
 Ask them to evaluate the usefulness of that measure.
For example, you could ask students to devise a measure for the "readability" of a piece of text. You might propose "average sentence length" as a starting measure then offer a number of texts for them to rank order using your suggested measure. They could then evaluate the usefulness of that measure using their own "gut feeling." You could use the following question to motivate this inquiry:
 Does text A feel more difficult than text B?
 What does the measure ignore?
They may then try to improve the measure by considering other factors such as average word length, average number of words or syllables per sentence, percentage of polysyllabic words, use of passive or active voice and so on. After arriving at their own individual measures, students may like to compare their own rank ordering of texts with each other in order to evaluate them.
For more formal assessment, you might choose a task from the materials available at the end of this CAT, then strip out a number of the steps which provide support in coming to a solution. In this way, you can assess the extent to which students have internalized these analytic skills.
Analysis
Student work can be measured against three criteria:
This generic scoring rubric may be modified and adapted for specific tasks.
Category of performance

Typical response

The student needs significant instruction

Student can use a given measure of a concept, but cannot identify its limitations.
Student cannot devise a different measure of the concept.

The student needs some instruction

Student can use a given measure of a concept and identify some of its limitations.
Student may suggest a different measure of the concept, but it may not be numerical.

The student's work needs to be revised

Student can use a given measure of a concept and identify its limitations.
The student can also formulate a precise numeric measure, but does not evaluate the measure or discuss where it is appropriate.

The student's work meets the essential demands of the task

Student can use a given measure of a concept and identify its limitations.
The student can also formulate a precise numeric measure and can evaluate the utility of that measure.

Malcolm Swan
Mathematics Education
University of Nottingham
Malcolm.Swan@nottingham.ac.uk
Most assessment practices seem to emphasise the reproduction of imitative, standardised techniques. I want something different for my students. I want them to become mathematicians  not rehearse and reproduce bits of mathematics.
I use the five 'mathematical thinking' tasks to stimulate discussion between students. They share solutions, argue in more logical, reasoned ways and begin to see mathematics as a powerful, creative subject to which they can contribute. Its much more fun to try to think and reach solutions collaboratively. Assessment doesn't have to be an isolated, threatening business.
Not just answers, but approaches.
Malcolm Swan is a lecturer in Mathematics Education at University of Nottingham and is a leading designer on the MARS team. His research interests lie in the design of teaching and assessment. He has worked for many years on research and development projects concerning diagnostic teaching (including ways of using misconceptions to promote long term learning), reflection and metacognition and the assessment of problem solving. For five years he was Chief Examiner for one of the largest examination boards in England. He is also interested in teacher development and has produced many courses and resources for the inservice training of teachers.
Jim Ridgway
School of Education
University of Durham
Jim.Ridgway@durham.ac.uk
Thinking mathematically is about developing habits of mind that are always there when you need them  not in a book you can look up later.
For me, a big part of education is about helping students develop uncommon common sense. I want students to develop ways of thinking that cross boundaries  between courses, and between mathematics and daily life.
People should be able to tackle new problems with some confidence  not with a sinking feeling 'we didn't do that yet'. I wanted to share a range of big ideas concerned with understanding complex situations, reasoning from evidence, and judging the likely success of possible solutions before they were tried out. One problem I had is that my students seemed to learn things in 'boxes' that were only opened at exam time. Thinking mathematically is about developing habits of mind that are always there when you need them  not in a book you can look up later.
You can tell the teaching is working when mathematical thinking becomes part of everyday thinking. Sometimes it is evidence that the ideas have become part of the mental toolkit used in class  'lets do a Fermi [make a plausible estimate] on it'. Sometimes it comes out as an anecdote. On graduate told me a story of how my course got him into trouble. He was talking with a senior clinician about the incidence of a problem in child development, and the need to employ more psychologists to address it. He 'did a Fermi' on the number of cases (wildly overestimated) and the resource implications (impossible in the circumstances). He said there was a silence in the group...you just don't teach the boss how to suck eggs, even when he isn't very good at it. He laughed.
Jim Ridgway is Professor of Education at the University of Durham, and leads the MARS team there. Jim's background is in applied cognitive psychology. As well as kindergarten to college level one assessment, his interests include the uses of computers in schools, fostering and testing higher order skills, and the study of change. His work on assessment is diverse, and includes, the selection of fast jet pilots, and cognitive analyses of the processes of task design. In MARS he has special responsibility for data analysis and psychometric issues, and for the CL1 work.
About MARS
The Mathematics Assessment Resource Service, MARS, offers a range of services and materials in support of the implementation of balanced performance assessment in mathematics across the age range K to CL1. MARS is funded by the US National Science Foundation, and builds on earlier funding which began in 1992 for the Balanced Assessment Project (BA) from which MARS grew.
MARS offers effective support in:
The Design of Assessment Systems: assessment systems are tailored to the needs of specific clients. Design ranges from the contribution of individual tasks, through to full scale collaborative work on test development, scoring and reporting. Clients include Cities, States, and groups concerned with educational effectiveness, such as curriculum projects and professional development initiatives.
Professional Development for Teachers: most teachers need help in preparing their students for the much wider range of task types that balanced performance assessment involves. MARS offers professional development workshops for district leadership and 'mentor teachers', built on materials that are effective when used later by such leaders with their colleagues in school.
Developing Design Skills: many clients have good reasons to develop their own assessment, either for individual student assessment or for system monitoring. Doing this well is a challenge. MARS works with design teams in both design consultancy and the further development of the team's own design skills.
To support its design team, MARS has developed a database, now with around 1000 interesting tasks across the age range, on which designers can draw, modify or build, to fit any particular design challenge.
Tell me more about this technique:
Mathematical Thinking CATs  Fault Finding and Fixing  Plausible Estimation
Creating Measures  Convincing and Proving 
Reasoning from Evidence
