New Traditions: Student Interview Protocol

John Moore
Department of Chemistry
University of Wisconsin-Madison
1101 University Ave.
Madison WI  53706
email: jwmoore@chem.wisc.edu
(608) 262-5154

Susan Millar
LEAD Center
University of Wisconsin-Madison
1402 University Ave.
Madison, WI  53706
email: smillar@engr.wisc.edu
(608) 265-5943

Debby Penberthy
LEAD Center
University of Wisconsin-Madison
1402 University Ave.
Madison, WI  53706
email: penberth@engr.wisc.edu
(608) 265-5927

Introduction:
In the middle of the Spring semester 1997, LEAD researchers interviewed a sample of 14 students from the UW-Madison's Topic-oriented Approach to Chemistry 104 (2nd semester general chemistry) using an open-ended interview protocol designed to elicit information on their experiences in the course overall and with particular course elements.  Students were identified using a “quasi” random process: that is, student names were generated using a random number to produce a list of students to contact. However, the actual list of interviewees was modified somewhat to ensure that the resulting group was similar to the make-up of the entire class and was comprised of at least one student of each of the eight TAs. The proportions of those interviewed were roughly similar (given the interview sample size of 14) to that of the entire lecture, with respect to gender and ethnic categories, incoming ability indicators, and academic rank. 

Conducting the interviews
The LEAD Center interview protocols were created on the basis of specific research questions. These research questions were often designed so as to determine the degree to which the course professor's learning goals were achieved and to examine the nature of the students' learning process. Thus, if a potential user of these instruments has different student learning goals, some of the interview questions may be less appropriate than others.  

All of the LEAD Center interview protocols included here are open-ended. In contrast to most surveys, open-ended interview protocols are designed to allow the respondent to speak freely about their experiences without being limited by predetermined response categories. Potential users of these instruments should keep in mind that the questions served as a guide to a “conversation” with the interview respondent; the questions were not asked verbatim or in the same order for each interview, as the interviewee was allowed to guide the conversation to some degree.  

Interviews were conducted by LEAD Center researchers. In all cases these researchers were considered “third party” in that they were not members of the academic departments through which the course or program under evaluation was taught. Thus, the identity of student interviewees was not revealed to the course instructors, and students were informed that this was the case before agreeing to participate. This was important because it allowed the student interviewees to feel that their opinions would not affect their relations with their course instructors. In cases where no third party evaluator is available or desirable, we recommend that the interviews be conducted by someone who has no control over the students' grades. One possibility is for two professors to interview each others' students. 

Analyzing the interviews
Sample results based on the use of LEAD Center interview protocols were obtained by analysis of verbatim transcripts of the audio-taped interviews. Although it is not necessary to transcribe the interviews, this does allow for more in-depth analysis of subtle themes. It is possible for the interviewer simply to take notes during the interviews and base their analysis on these notes. 

The LEAD Center analysis process consists of developing inductive, thematic codes for the students' experiences. Generally, the researchers first read all of the transcripts. Then, they develop a rough coding scheme based on themes which appear in multiple interviews. Using this scheme, each individual transcript is coded. Then, interview excerpts are cut and pasted into an analysis document so that statements which are alike can be examined side by side. (Alternatively, a qualitative research software package may be used to sort and organize interview themes.) The coding scheme is gradually refined so that it represents a listing of the umbrella themes that capture the range of experiences discussed in the interviews. The analysis process is quantitative only in the sense that rough counts of the number of interviewees who spoke about a particular type of experience are developed. It differs from the traditional definition of quantitative analysis in that it does not tell one how many of the participants had a particular experience, but only how many chose to talk about it in the interview. Again, the interviewees are engaged in conversation, and therefore, not all issues are touched upon in each interview. Thus, though a small number of students may discuss an issue, the issue might be important in the experiences of a larger number or percentage of students. 

___________________________________________________________

New Traditions: Student Interview Protocol

Introductions and presentation of the Informed Consent form.

Briefly review points from the Informed Consent, focusing on how the evaluation and assessment process relies on learning about what and how well students are learning in this course. The evaluators are allies who can help students communicate things to the instructors that might be awkward for students to say directly. Check that the student is comfortable with the tape recorder. If not, just take notes. 

Background

1.	Please tell me a little about where you are from and what type of high school you attended.
	
2.	Tell me about your preparation for college math and science courses. [Prompt: If someone says that they are either ahead or behind other people, ask, "How do you know?"]
	
3.	Tell me a little about ideas you have for a major, or possible plans for your future professional life. Have your plans for a major changed since last semester?
	
4.	Tell me a little about your experience in Chem 103. How did it go? Who was your professor?

Chemistry 103
	
5.	Why did you register for [this section of]'s 104? [Probe for whether or not they knew about the teaching innovations.]
	
6.	What were your first impressions of Chem 104? [Probe for whether or not they were surprised or distressed by the groupwork or other differences.]
	
7.	If you had to specify one thing that is helping you learn chemistry the most this semester, what would it be? Why?

Lecture-related questions
	
8.	Can you describe the lecture section of your class? What does the professor do, what do you do?
	
9.	a) Do you think that your lecture section is an effective way for you to learn chemistry? Please explain.
	
	b) Is there any aspect of the lecture section that you particularly like or think is particularly effective for you? Can you explain and give an example?
	
	c) Is there any aspect of the lecture section that you particularly dislike or think is not effective for you? Can you explain and give an example?
	
10.	Tell me what you think about the participation element of the lectures: when [Professor X] asks you to either raise your hand and answer a question or discuss the question with your neighbor.
	a) How, if at all, do they help you learn chemistry?
	
	b) Are there any other benefits or detriments to engaging in this activity?

Laboratory
	
11.	Can you describe the laboratory section of your class?
	
12.	What, if anything, are you learning? [Probe for lab technique, concepts, etc.]
	
13.	Do you think that the laboratory section is an effective way for you to learn chemistry? Please explain.
	
14.	Is there any aspect of the lab that you particularly like or think is particularly effective for you? Can you explain and give an example?
	
15.	Is there any aspect of the lab section that you particularly dislike or think is not effective for you? Can you explain and give an example?
	
16.	Can you describe the labs? [Probe from cookbook vs. inquiry-based.]
	
17.	Tell me about the integrated lab reports. How, if at all, do they help you learn chemistry?
	
18.	How, if at all, does the physical environment in the lab affect your experience in lab either negatively or positively?

Discussion Section
	
19.	Can you describe your discussion section? What typically happens during discussion?
	
20.	Do you think that the discussion section is an effective way for you to learn chemistry? Please explain.
	
21.	Is there any aspect of the discussion section that you particularly like or think is particularly effective for you? Can you explain and give an example?
	
22.	Is there any aspect of the discussion section that you particularly dislike or think is not effective for you? Can you explain and give an example?

Topic-oriented Approach
	
23.	The professor in this course is using what he calls a "topic-oriented approach" to teaching.  What do you think of this approach? 
	a) Are there positive effects for you? 
	
	b) Negative? [Prompt for effects on learning and on attitude.]
	
24.	Do you notice other differences in this professor's approach that you would like to comment on?
	
25.	What is your reaction to the nature of the quizzes, problem sets and exams? Integrated lab reports? [Probe for their reaction to the shift from doing calculation-based problems to writing out their answers in essay form and addressing concepts.]

Roles
	
26.	a) How important is the TA to your learning of chemistry in this course?
	
	b) The professor?
	
	c) Other students?

Workshops
	
27.	Have you attended any of the workshops?
	a) If not, why not? [Prompt for what might make them to go.]
	
	b) If so...Tell me what it was like.
	i. What did you work on?
	ii. Did you go with a group of other students?
	iii. How did you interact with the other students in your group? With the TA?
	iv. Did you go to the workshop with your own TA or with another TA? [If another TA]: What was your reaction to working with a new TA?
	v. What, if anything, did you get out of it?
	vi. Why did/do you go?

General Questions
	
28.	Has taking this course changed the way you think about chemistry?
	
29.	Is there anything else that you can tell us to help us understand your experience in this class?
	
30.	Do you have any questions for me?