Including Affective Behavior in Course Grades Barry McNeill, Veronica Burrows, and Lynn Bellamy College of Engineering and

Applied Science Arizona State University Abstract The authors published an article in the October issue of ASEE Journal of Engineering Education in which they described an assessment process based on using only three outcomes (exceeding expectations, meeting expectations, or needing improvement). The paper showed how the assessment process could be implemented with the use of checklists. The paper dealt essentially with assessing the cognitive behavior of the students and did not address affective behavior. Affective behavior (e.g., coming to class, turning work in on time, etc.) has always played some role in establishing a final grade and in fact some classes actually have course outcomes that are purely affective in nature (e.g., value the use of computer tools). To better account for the affective behavior aspects, the authors have developed an assessment method that assesses the cognitive and affective behaviors separately and then combines them to generate a single course grade. The paper will present background on affective behavior, present arguments why cognitive and affective assessment should be done separately, will show one way to assess affective behavior using evidence of self regulation, and concludes with a process that shows how to combine affective and cognitive assessment into a course grade. Introduction In the early 1950s a group of educational psychologists addressed the problem of defining and assessing the various levels of mastery associated with learning. These psychologists divided the problem into three behavioral domains: the cognitive, dealing with the recall or recognition of knowledge and the development of intellectual abilities and skills; the affective, dealing with interests, attitudes, appreciations, values, and emotional sets or biases; and, the psychomotor, dealing with the manipulative or motor-skill area. The result of this work was the creation of two taxonomies, one for cognitive behavior [1] and one for affective behavior [2]. Cognitive based learning objectives have always been part of a course's learning objectives. After all, cognitive objectives are the ones directly related to achieving the knowledge and skill needed to solve problems. But what about the affective domain, are there affective learning objectives? Does affective behavior get assessed? Consider how a homework assignment is typically assessed: 1. the assignment is reviewed to determine what parts are correct and what parts are not 2. points are either assigned for the correct parts or removed for the wrong parts 3. a final point count is obtained by either adding up the points for correct work or adding up the points for wrong work and subtracting from the total possible

4. the final point count is then adjusted for such things as: submitted late (minus 10 points or even no acceptance), student had a bad day (drop worst homework assignment), the student did not participate in the class discussion (minus 10 points), etc. The first three steps involve assessing cognitive issues; the last step involves affective behavior. Thus faculty do assess affective behavior whether they consciously think about it or not. Engineering faculty rarely explicitly state these desired affective behaviors in the context of course expectations. In new courses, where such affective outcomes are often explicitly described, research has shown that these courses tend to loose their affective objectives over time [2]. Failure to explicitly consider affective objectives is going to change. New ABET requirements in the EC2000 accreditation process require that some courses have explicitly stated affective objectives that are assessed (e.g., ability to function on a multidisciplinary team, recognition of the need for life long learning are required are affective outcomes required by EC 2000). EC 2000 will force engineering faculty to more clearly define the affective behaviors they wish to encourage and to have a method for assessing these behaviors. Is the combining of affective and cognitive performance, as outlined above, an appropriate way to assess student performance? We dont believe it is and we suggest there is a better, more uniformly consistent way to handle both cognitive and affective assessment. This paper will discuss problems with the current method of handling affective behavior, will describe a method for assessing one aspect of affective behavior, (i.e., self regulation), and will conclude by suggesting a way to include affective assessments in a final course grade. What is Wrong with the Current Affective Assessment? There are several problems with the traditional point-based assessment processes. First, changes in affective and cognitive behaviors are not linked [2]. Achieving the desired change in the cognitive domain does not ensure or guarantee desired changes in the affective domain. It is even possible to accomplish cognitive objectives at the expense of affective objectives. To accomplish affective objectives it is important to have specific learning experiences related to the affective objective, just as is done for the cognitive objectives. This de-coupling of these two objectives strongly suggests they be assessed separately. A second problem with traditional assessment approaches is that combining affective and cognitive assessment impairs the cognitive assessment of the work. For example, if work is not accepted as a consequence of an affective failure (e.g., lateness), it is not possible to assess how well or poorly the student has mastered some cognitive objective. If work is accepted but points are taken off, the grade does not accurately indicate the cognitive performance. In either case the student does not really get a complete clear assessment of either the cognitive effort or the affective effort. There are several less important problems with current methods as well. Current methods are often used with no explicit statement to the students concerning the type of behavior the course is trying to encourage. The idea of what is acceptable affective behavior may be implied but not stated. Finally, as is true of many assessment processes that use points, the

method is rather arbitrary. For example, on what educational basis is ten points the right number of points to reflect the seriousness of the poor affective behavior? Assessment of Affective Objectives The first step in assessment is to define objectives. Affective objectives can range from very low level commitment (a willingness to receive information) to higher levels where students demonstrate that they value the knowledge and skills they are gaining. We believe that it is appropriate to choose overall objectives that fall between these two extremes. We have defined affective behavior objectives related to the willingness of the students to respond (i.e., to do what we ask them to do). We define desired affective behavior under the title of self-regulation. what is self-regulation Self regulation is a term used extensively in the current literature. Zimmerman [3] writes: Since the founding of the republic, American educational leaders have stressed the importance of individuals assuming personal responsibility and control for their own acquisition of knowledge and skill. He goes on to describe the self regulated student as follows: They approach educational tasks with confidence, diligence, and resourcefulness. ... Self-regulated learners are aware when they know a fact or possess a skill and when they do not. ... Self-regulated students pro-actively seek out information when needed and take steps to master it. When they encounter obstacles such as poor study conditions, confusing teachers, or abstruse text books, they find a way to succeed. Some characteristics of self regulated learning are: Self-regulated learners plan, set goals, organize, self-monitor, and self-evaluate at various points during the process of acquisition. Self-regulated learners monitor their strategies and modify as appropriate (i.e., they utilize self-oriented feedback). Self-regulated learning involves more than a capability to execute a learning response by oneself (i.e., self-control) and more than a capability to adjust learning responses to new or changing conditions from negative feedback. It involves proactive efforts to seek out and profit from learning activities. At this level, learners are not only selfdirected in a metacognitive sense but are self-motivated as well.

establishing affective objectives Given that we want students to become self-regulated (one part of life long learning, an EC 2000 required outcome), we have defined the following affective objectives for our freshman engineering class: 1. students will come to class on time 2. students will submit assignments on time

3. students will participate positively in class activities (in course materials we usually define this even more explicitly, with specific sub-objectives for positive communication, positive use of resources, and academic honesty) 4. a student will correct technical errors in their work on their first attempt 5. For upper division classes, we additionally expect that students will submit some fraction of their work that is technically correct on the first submission These are behaviors that most faculty feel students should demonstrate, but most likely objectives that have not been articulated as explicit course objectives. defining affective learning experiences As was mentioned above, if we want to actually have the students achieve affective objectives there must be learning experiences that permit this achievement. Once the objectives are defined it is relatively easy to supply an activity that lets students demonstrate mastery of the objective. Most of the objectives listed above can be demonstrated in the normal course of class operation. Only in a class that had no assignments and only lecture would some of the affective objectives listed above not be easily demonstrable by the students. assessing affective behavior In much the same we have earlier suggested using checklists to assess cognitive behavior [4], we suggest affective behavior can also be assessed with a checklist. The earlier paper broke cognitive assessment down into three categories of elements to assess: expected features (these must be present and are assessed as yes/no), revealed features (these are discussed as desirable and are assessed as Wow, OK, and Weak), and exciting features (unexpected elements of performance that add significantly to the quality of the work).

Affective performance is most like the expected features, a student either demonstrates the desired behavior or not. The checklists that we now use include both cognitive and affective assessment. The affective behavior (i.e., self regulation) issues appear at the top of the checklist. A checklist used to assess student performance on a quiz is shown in Figure 1. For this quiz there are three major self-regulation issues. First, the quiz must be turned in on time (item A in Figure 1). Second, the student must use the correct checklist (item B in Figure 1). Finally there are two items related to the correct completion of the headings on the checklist and the ParScore form1 (items C and D). This checklist is one that comes

1 The ParScore form is the answer sheet form used to score the quiz.

Name: ________________________ 1. Work Evaluated by

Team______________________________ Date

Assessment Symbol/Color

You are to complete only the top line. The grader/instructor will complete this checklist
Yes No Self Regulation Issues (completed by instructor &/or grader) A. The quiz was submitted on time (if No, instructor initial under No)

B. The ParSCORE form was stapled to the correct quiz checklist

C. The top line of the quiz checklist was completed correctly D. The ParSCORE form headings were correctly and completely done E. There were no other Self Regulation Problems (describe if no_________________) Enter a 1 (one) in the cell to left if any of the items listed above are marked as No Comments on Self Regulation Faults

Yes

No

Summary of Quiz Journal Performances An engineering journal was submitted If journal submitted, the engineering journal met expectations (see Journal subChecklist)

Enter the percent of quiz questions correctly answered in the box to the left

Results of Quiz Assessment

Each of these definitions uses the term satisfies Performance requirements. The quiz Performance requirements are defined in Table 2 of the first day materials for Concepts (yellow). E, satisfies Performance requirement for E M, satisfies Performance requirement for M NI, satisfies Performance requirement for NI NS, no quiz submitted

NI

NS

SRFault

If the quiz was assessed as either NI or NS you may submit, one week after this is returned, either a Quiz Correction Assignment (for an NI) or A Quiz Make-up assignment (for an NS). Successful completion of these assignments (i.e., a Meets) will change your quiz grade to Meets.

Figure 1 Sample Quiz Checklist Showing how Affective Behavior is Assessed

relatively late in the course and has more self regulation issues that the ones used early in the course. The checklist for the first quiz only had item A2. If any of the items A to D not true, they are marked as No and the student is assessed a Self-Regulation Fault. The Self Regulation Fault has no affect on the assessment of the quiz; the quiz can still be assessed as Exceeds expectations. We keep track of the number of Self Regulation Faults a student accumulates over the semester and how these faults are used in determining a course grade is discussed below. Advantages Of Separating Cognitive And Affective Assessments There are a number of pedagogical advantages in separating cognitive and affective assessment but they all come back to the main reason for separating them: separation allows us to work directly and explicitly with the appropriate behavior (cognitive or affective). If a student is doing high quality cognitive work but is having difficulty getting the work done on time, we can now recognize (and "reward") the high quality work and at the same time keep track of self regulation problems. We can then, for example, discuss with the student strategies that might help them get their work done on time. Course Grade Determination If we separate affective and cognitive assessment for each assignment how are these two assessments eventually re-combined? The combination method we use assumes that nobody is free of self regulation faults, e.g., everyone will miss a deadline at some time or other. We expect all students to get some self regulation faults during the semester. While everyone will probably accrue a few faults we feel that an A student would not miss too many deadlines or not complete the quiz heading correctly too many times. Further, a student who eventually gets all work done technically correctly but has to resubmit the work many times to get the work right is probably not a B student. an algorithm for combining affective and cognitive assessments We have incorporated these ideas into the following process for determining a course grade. 1. Determining A Course Cognitive Grade. The first step is to determine the cognitive course grade. This is done as you would normally determine grades (add up points, count number of exceeds expectations, etc.) 2. Determine the Affective Course Grade. The second step is to add up all the Self Regulation Faults that the student has accumulated over the course of the semester. 3. Determine the Course Grade The third step combines the affective and cognitive assessments using a mapping similar to that shown in Table 1.

2 The changing of the checklist over the course of the semester is an example of the progressive nature of

the checklists used in a course.

Self-Regulation Assessment (number of SR-Faults) 03 Cognitive Assessment A B C D E 45 68 >8

A B C D E

B B C D E

C C C D E

D D D D E

Table 1 Mapping Table Used to Combine Cognitive and Affective Assessment into a Single Course Grade The mapping shown in Table 1 is similar to one used in the first year class. Items of interest about the mapping are: 1. As long as the number of Self-regulation Faults does not exceed three, the Course grade matches the Cognitive Course grade (see first column). 2. A student who accumulates more than three faults cannot get an A in the course, no matter how outstanding their technical performance. 3. There is no way self regulation can cause a student to fail the course (see last column in Table 1). The only way to fail a course is to fail it because of cognitive problems. some experiences in using the system. We introduced this system in our classes with some trepidation. After all we did not really know what a reasonable number of faults might be to distinguish the A, B, C, and D student. We were worried that if we set the number of free faults (the ones that do not negatively impact your grade) too high we would not encourage the type of behavior we wanted. If we set the number of free faults to low we would be putting too much emphasis on the affective behavior. The number of faults used in Table 1 to distinguish between the various grades, depends on a number of issues. The total number of assignments must be taken into account; the larger the number of assignments the higher the free faults should be. Also important is how far from traditional learning environments the class has moved. In classes that use active learning and new assessment methods (e.g., checklist) there will be self regulation issues related to becoming familiar and comfortable with the changes in the learning environment. We have used this method in the first year course a third year design, and in a materials course. Out of over 400 grades determined there have only been a handful (on the order of 15) of course grades that did not match the cognitive assessment. In other words, a small fraction of students accumulate many self regulation faults, but those students are the B and C students, cognitively.

Summary This paper showed that all faculty assess affective behavior is some fashion. Research strongly suggests that affective and cognitive behavior are not closely coupled and should be defined and assessed separately. A method of assessing affective objectives using self regulation was presented. The paper ended with a process for combing the affective assessments and cognitive assessments at the end of the semester into a final course grade. References [1.] Bloom, B.S., editor, Taxonomy of Educational Objectives Book 1 Cognitive Domain, ISBN: 0-582-280109-9, Longman, 1984. [2.] Krathwohl, D.R., Bloom, B.S., and Masia, B.B., Taxonomy of Educational Objectives Book 2 Affective Domain, ISBN: 0-582-28239-X, Longman, 1964. [3.] Zimmerman, B.J., Self-Regulated Learning and Academic Achievement: An Overview, Educational Psychologist, 25, 1990, (1), pp. 3-17 [4.] McNeill, B., Bellamy, L, Burrows, V. A Quality Based Assessment Process For Student Work Products, Journal of Engineering Education, Vol. 88, No.4, October 1999, pp. 485-500.