Planning Formative Assessment Workshop PDF

Planning
Formative Assessment Workshop 2012

Planning Formative
Assessment Workshop
3:30‐4:30pm, CCIS L2‐200, Jan. 12, 2012

Based on a follow up workshop to a 2008 TLEF Award:
“Formative Assessment Technique Development in
Engineering”

Planning Formative Assessment Workshop 2012

Speaker contact info: John A. Nychka (Assoc. Prof., CME); jnychka@ualberta.ca; 492‐4796; ECERF 7‐040

This workshop is based on a follow up workshop to an initial University of Alberta Teaching Learning
and Enhancement Fund (TLEF) Award: “Formative Assessment Technique Development in
Engineering” (2008)
J.A. Nychka, S. Kresta, U. Sundararaj, Q. Liu, F. Forbes,
Chemical and Materials Engineering, Faculty of Engineering; $20,000

Excellence in education cannot be achieved or continue without assessment. Assessment is not just
grade reporting. Assessment involves the measure of progress to help students and instructors learn
how to improve. We will use TLEF funds to:
a. hold a classroom assessment techniques workshop by an expert leader in the field;
b. assist University of Alberta faculty become formally trained in the techniques of formative
assessment; (through workshops like this one!)
c. promote culture change from teacher‐centered instruction to learner‐centered instruction;
d. give instructors the tools and forum necessary to develop their own formative assessment
techniques and incorporate them in course re‐design;
e. encourage and facilitate instructors to share their research on formative assessment in
scholarly venues (i.e. journals).

AGENDA
3:30 to 3:35pm Welcome and Introduction
3:35 to 3:42pm Part 1: Brief overview of “Learner Outcomes” and “Formative Assessment”
3:42 to 4:25pm Part 2: “Planning Formative Assessment”
4:25 to 4:30pm Feedback and closing remarks

Not all the activities in this handout will be covered during the workshop, but working through them
with some colleagues (especially those outside of your department or faculty) will be very useful.

IMPORTANT!
The workshop is meant to be interactive. In preparation for the workshop, having one particular
course in mind (a course you have taught or will teach in the upcoming academic year):
1) Take the online teaching goals inventory (with respect to the chosen course) and bring the
results with you to the workshop. The teaching goals inventory can be accessed at:
http://fm.iowa.uiowa.edu/fmi/xsl/tgi/data_entry.xsl?‐db=tgi_data&‐lay=Layout01&‐view
2) Bring the course syllabus.
3) Bring an exam from the course.
4) Bring a homework or lab assignment from the course.
5) Identify a topic or unit within the course which you can easily assess, and another which you
have difficulty assessing.

References and Further Reading
Some of the resources used today for the “Outcomes” and “Assessment” sessions were modified
versions which were originally prepared and supplied by the Centre for Teaching and Learning, and can
be found at: http://www.ctl.ualberta.ca/Teaching_Services/TES_Docs/TES_Resources.html

Prior TLEF efforts on Formative Assessment:
http://www.ctl.ualberta.ca/Symposiums/LessTeachingMoreLearning.html
(Including Sadler keynote podcast, papers, and workshop materials; Angelo keynote podcast,
workshops I‐III, and reference materials).

Outcomes
Teaching Goals Inventory – online system
http://fm.iowa.uiowa.edu/fmi/xsl/tgi/data_entry.xsl?‐db=tgi_data&‐lay=Layout01&‐view

Portland State University – “Assessment Step‐by‐Step”
http://www.pdx.edu/cae/assessing‐teaching‐learning
http://www.pdx.edu/cae/assessment‐step‐step

Lincoln Land Community College – “Bloom’s Taxonomy of Measurable Verbs”
http://www.llcc.edu/LinkClick.aspx?fileticket=%2F0BA4qlDaAE%3D&tabid=3938

Kansas State University – “How to Write Student Learning Outcomes” (has a good verb listing)
http://www.k‐state.edu/assessment/slo/how.htm

National Center for Geographic Information and Analysis, University of California Santa Barbara
(condensed verb listing)
http://www.ncgia.ucsb.edu/education/curricula/giscc/units/format/outcomes.html

Prairie South School Division, Moose Jaw, SK – Student Learning Outcomes
http://www.prairiesouth.ca/staff/curriculum‐a‐learning/assessment/formative‐assessment‐afl.html

American Psychological Association – The Assessment Cyberguide for Learning Goals and Outcomes,
2009: http://www.apa.org/ed/governance/bea/assessment‐cyberguide‐v2.pdf

Formative Assessment/Assessment/Classroom Assessment
Thomas A. Angelo and K. Patricia Cross (1993), “Classroom Assessment Techniques: A Handbook for
College Teachers”, 2nd Ed., Jossey –Bass, San Francisco, pp. 427.

Carol Boston (2002) “The Concept of Formative Assessment” Practical Assessment, Research &
Evaluation, 8(9) http://pareonline.net/getvn.asp?v=8&n=9


Prairie South School Division, Moose Jaw, SK – Classroom Formative Assessment
http://www.prairiesouth.ca/staff/curriculum‐a‐learning/assessment/formative‐assessment‐afl.html

Valerie J. Shute (2007), “Focus on Formative Feedback” Educational Testing Service Research Report
http://www.ets.org/Media/Research/pdf/RR‐07‐11.pdf

McGill University Office of Student Teaching: Formative Assessment Rubrics Guide for evaluation of
student teaching: http://www.mcgill.ca/files/ost/RUBRICS_GUIDE.pdf

Lincoln University ‐ “Avoiding Assessment Pitfalls”
http://www.lincoln.edu/assessment/planning/pitfalls.pdf

University of Kentucky – Classroom Assessment Techniques (a variety of examples of created CATs)
http://www.uky.edu/IRPE/assessment/classroomtech.html

“9 Principles of Good Practice for Assessing Student Learning” American Association for Higher
Education (2003): http://www.cnu.edu/assessment/pdf/aahe_9_assessment_principles.pdf

Peter T. Ewell (2006) “Improving Undergraduate Education through the Assessment of Student
Learning (Presentation)” National Center for Higher Education Management Systems:
http://www.nchems.org/pubs/detail.php?id=84

University of Illinois, Urbana‐Champagne – “Unit Plan for Assessing and Improving Student Learning
in Degree Programs”. Multiple discipline plans are available:
http://cte.illinois.edu/outcomes/unit_assess.html


Activities

Planning Formative
Assessment


Activity 1: Background Knowledge Probe

Circle the answer that most represents your current state of knowledge:

1) Classroom Assessment
a) Have never heard of this
b) Have heard of it, but don’t really know what it means
c) Have some idea what this means, but not too clear
d) Have a clear idea of what this means and can explain it

2) Teaching Goals Inventory
a) Have never heard of this
b) Have heard of it, but don’t really know what it means
c) Have some idea what this means, but not too clear
d) Have a clear idea of what this means and can explain it

Activity 2: Focused Listing
On the lines below, please list 5‐7 words or short phrases that describe/define what the phrase
“Classroom Assessment” means to you.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________

Activity 3: Pro and Con Grid
Directions: Considering everything you know about CATs at this point, what do you see as the most
important pros (cons), or costs (benefits) of using this approach. List at least 3 important cons (costs)
and at 3 pros (benefits) below.
Advantages
of CATS Disadvantages of CATS

Planning
P hop 2012
g Formattive Asseessmentt Worksh

Activity 4: A
Applicatio
ons Card
Adapted fro
om Angelo: http:://www.ctl.ualbe ents/WorkshopI..pdf
erta.ca/docume

Planning
P hop 2012

Samp
ple Assesssment Prroject Plaan

Angelo
o and Cross (1993) pg 43‐44.
Planning
P hop 2012

The Asses
T ssment P
Planner

Angelo
o and Cross (1993) p 49.
Activvity 5: What woulld you do
o?
Anaylze the proposed assessme ent plans in th
he sample ca lculus examp
ple above, payying close
attenttion to determ
mining answe
ers for questions 5‐10 from
m the above AAssessment PPlanner.

Planning
P hop 2012

5.

Angelo: http
p://www.ctl.ualb
berta.ca/documents/WorkshopI.pdf


Sample CAT: Customized Content, Form, and Function Outlines
University of Kentucky MSE 201 Fall 2006 CAT 11: 7 minutes October 13, 2006
This is a Classroom Assessment Technique, not a quiz!
You are supposed to tell your boss how long to process some flat 1018 steel plate in a carburization
furnace. He tells you that the plate will be in a furnace at a temperature of 893K in an atmosphere of
carbon monoxide/carbon dioxide (equivalent carbon concentration of 0.8wt%). He would like the
treatment to achieve a carbon content of 0.34wt% at a depth of 200 microns. The diffusivity of carbon in
steel at the temperature in question is 2.44x10‐11 m2/s.
Circle the equation(s) you can use to solve this problem.
⎛ ∂C ⎞
A. J = − D⎜ ⎟
⎝ ∂x ⎠
⎛ ∂C ⎞ ⎛ ∂ 2C ⎞
B. ⎜ ⎟ = D ⎜⎜ 2 ⎟⎟
⎝ ∂t ⎠ ⎝ ∂x ⎠
C. xeff ≅ γ Dt
⎛ C ( x, t ) − C o ⎞ ⎛ x ⎞
D. ⎜⎜ ⎟⎟ = 1 − erf ⎜⎜ ⎟⎟
⎝ C s − Co ⎠ ⎝ 2 Dt ⎠
Using the information you have in the problem statement fill in the grid below:
Variable Definition Units Value from problem

x

t

Cs

Co

C(x,t)

D

γ

Nychka: http://www.uky.edu/IRPE/assessment/classroomtech.html

Activity 6: CAT Analysis
For the previous sample CAT, the following higher order learning skills were primarily targeted:
1. develop ability to synthesize and integrate information and ideas
2. problem solving skills
3. analytic skills

along with the following teaching goals:
4. improve ability to follow directions, instructions and plans
5. learn to evaluate methods and materials in this subject

Notes: Three of the four equations listed can be used to solve the problem.

The data collected from the CAT is listed below:
16 students handed in the CAT (out of 22 registered students)
5 students circled only 1 equation (100% chose 1 of the 3 correct answers)
7 students circled 2 equations (86% chose 2 of the three correct answers)
4 students circled 3 equations (75% chose 3 of 3 correct answers)

10 students had 4 or less blank spaces out of 21
6 students did not attempt to fill in 11 or more blank spaces out of a total possible 21

Figure 1. a) Number of students as a function of unanswered spaces; b) Mean number of correct
answers, Nc, versus the number of spaces left unanswered. Error bars are one standard deviation.

How would you interpret the collected data with respect to the teaching goals?
Is the data meaningful?
What would you do to change the CAT to make it more effective?

Angelo and Cross (1993) pg 59.

50 CATS by Angelo and Cross (1993)
Prepared by D. Moore & K. Cunningham, September 2006
University of Kentucky, Lexington
Source: http://www.uky.edu/IRPE/assessment/classroomtech.html

Techniques for Assessing Course‐Related Knowledge & Skills

I. Assessing Prior Knowledge, Recall, and Understanding
The CATS in this group are recommended to assess declarative learning, the content of a particular
subject.
1. Background Knowledge Probe: short, simple questionnaires prepared by instructors for use at the
beginning of a course or at the start of new units or topics; can serve as a pretest; typically elicits
more detailed information than CAT2. .
2. Focused Listing: focuses students’ attention on a single important term, name, or concept from a
lesson or class session and directs students to list ideas related to the “focus.”
3. Misconception/Preconception Check: focus is on uncovering prior knowledge or beliefs that
hinder or block new learning; can be designed to uncover incorrect or incomplete knowledge,
attitudes, or values
4. Empty Outlines: in a limited amount of time students complete an empty or partially completed
outline of an in‐class presentation or homework assignment
5. Memory Matrix: students complete a table about course content in which row and column
headings are complete but cells are empty
6. Minute Paper: perhaps the most frequently used CAT; students answer 2 questions (What was the
most important thing you learned during this class? And What important question remains
unanswered?)
7. Muddiest Point: considered my many as the simplest CAT; students respond to 1 question (What
was the muddiest point in _________ ?); well suited to large, lower division courses but not to
those which emphasize integration, synthesis and evaluation

II. Assessing Skill in analysis and Critical Thinking
The CATS in this group focus on analysis—the breaking down of information, questions, or problems to
facilitate understanding and problem solving
8. Categorizing Grid: student complete a grid containing 2 or 3 overarching concepts and a variety of
related subordinate elements associated with the larger concepts
9. Defining Features Matrix: students categorize concepts according to presence or absence of
important defining features
10. Pro and Con Grid: students list pros/cons, costs/benefits, advantages/disadvantages of an issue,
question or value of competing claims
11. Content, Form, and Function Outlines: in an outline form, students analyze the “what” (content),
“how” (form), and “why” (function) of a particular message (e.g. poem, newspaper story,
billboard, critical essay); also called “What, How, & Why Outlines
12. Analytic Memos: students write a one‐ or two‐page analysis of a specific problem or issue to help
inform a decision‐maker

III. Assessing Skill in Synthesis and Creative Thinking
The CATS in this group focus on synthesis—each stimulate the student to create, and allow the faculty to
assess, original intellectual products that result from a synthesis of course content and the students’
intelligence, judgment, knowledge, and skills.
13. One‐Sentence Summary: students answer the questions “Who does what to whom, when, where,
how, and why?” (WDWWWWHW) about a given topic and then creates a single informative,
grammatical, and long summary sentence
14. Word Journal: involves a 2 part response; 1st the student summarizes a short text in a single word
and 2nd the student writes 1‐2 paragraphs explaining the word choice
15. Approximate Analogies: students simply complete the 2nd half of an analogy—a is to b as x is to y;
described as approximate because rigor of formal logic is not required
16. Concept Maps: students draw or diagram the mental connections they make between a major
concept and other concepts they have learned
17. Invented Dialogues: students synthesize their knowledge of issues, personalities, and historical
periods into the form of a carefully structured illustrative conversation; 2 levels of invention
(select and weave quotes from primary sources or invent reasonable quotes that fit characters
and context)
18. Annotated Portfolios: students assemble a very limited number of examples of creative work and
supplement with own commentary on significance of examples

IV. Assessing Skill in Problem Solving
The CATS in this group focus on problem solving skills of various kinds—recognition of types of problems,
determining principles and techniques to solve, perceiving similarities of problem features and ability to
reflect and then alter solution strategies.
19. Problem Recognition Tasks: students recognize and identify particular problem types
20. What’s the Principle?: students identify principle or principles to solve problems of various types
21. Documented Problem Solutions: students track in a written format the steps they take to solve
problems as if for a “show & tell”
22. Audio‐ and Videotaped Protocols: students work through a problem solving process and it is
captured to allow instructors to assess metacognition (learner’s awareness of and control of
thinking)

V. Assessing Skill in Application and Performance
The CATS in this group focus on students’ abilities to apply important—sometimes referenced as
conditional knowledge—knowing when and where to apply what know and can do.
23. Directed Paraphrasing: students paraphrase part of a lesson for a specific audience demonstrating
ability to translate highly specialized information into language the clients or customers can
understand
24. Application Cards: students generate examples of real‐work applications for important principles,
generalizations, theories or procedures
25. Student‐Generated Test Questions: students generate test questions and model answers for
critical areas of learning
26. Human Tableau or Class Modeling: Students transform and apply their learning into doing by
physically modeling a process or representing an image.

27. Paper or Project Prospectus: Students create a brief plan for a paper or project based on your
guiding questions.

Techniques for Assessing Learner
Attitudes, Values, and Self‐Awareness

VI. Assessing Students’ Awareness of Their Attitudes and Values
The CATS in this group are designed to assist teachers in developing students’ attitudes, opinions, values,
and self‐awareness within the course curriculum.
28. Classroom Opinion Polls: Students indicate degree of agreement or disagreement with a
statement or prompt.
29. Double‐entry Journals: Students record and respond to significant passages of text
30. Profiles of Admiral Individuals: Students write a brief description of the characteristics of a person
they admire in a field related to the course
31. Everyday Ethical Dilemma: Students respond to a case study that poses a discipline‐related ethical
dilemma
32. Course‐related Self‐Confidence Surveys: Students complete an anonymous survey indicating their
level of confidence in mastering the course material

VII. Assessing Students’ Self‐Awareness as Learners
The CATS in this group are recommended to help students express personal goals and clarify self‐concept
in order to make a connection between the articulated goals and those of the course.
33. Focused Autobiographical Sketches: Students write a brief description of a successful learning
experience they had relevant to the course material.
34. Interest/Knowledge/Skills Checklists: Students complete a checklist survey to indicate their
knowledge, skills and interest in various course topics.
35. Goal Ranking and Matching: Students list and prioritize 3 to 5 goals they have for their own
learning in the course.
36. Self‐Assessment Ways of Learning: Students compare themselves with several different “learning
styles” profiles to find the most likely match.

VIII. Assessing Course‐Related Learning and Study Skills, Strategies, and Behaviors
The CATS in this group focus both student and teacher attention on the behaviors the student actually
engages in when trying to learn.
37. Productive Study‐Time Logs: Students complete a study log to record the quantity and quality of
time spent studying for a specific course.
38. Punctuated Lectures: Students briefly reflect then create a written record of their listening level of
a lecture. Repeat twice in the same lecture and 2‐ 3 times over 2 to 3 weeks.
39. Process Analysis: Students outline the process they take in completing a specified assignment.

40. Diagnostic Learning Logs: Students write to learn by identifying, diagnosing, and prescribing
solutions to their own learning problems.

Techniques for Assessing Learner
Reactions to Instruction

IX. Assessing Learner Reactions to Teachers and Teaching
The CATS in this group are designed to provide context‐specific feedback that can improve teaching
within a particular course.
41. Chain Notes: On an index card that is distributed in advance, each student responds to an open‐
ended prompt about his or her mental activity that is answered in less than a minute.
42. Electronic Survey Feedback: Students respond to a question or short series of questions about the
effectiveness of the course.
43. Teacher‐designed Feedback Forms: Students respond to specific questions through a focused
feedback form about the effectiveness of a particular class session.
44. Group Instructional Feedback Technique: Students respond to three questions related to the
student’s learning in the course.
45. Classroom Assessment Quality Circles: A group or groups of students provide the instructor with
ongoing assessment of the course through structured interactions.

X. Assessing Learner Reactions to Class Activities, Assignments, and Materials
The CATS in this group are designed to give teachers information that will help them improve their course
materials and assignments.
46. RSQC2 (Recall, Summarize, Question, Connect and Comment): Students write brief statements
that recall, summarize, question, connect and comment on meaningful points from previous class.
47. Group‐Work Evaluation: Students complete a brief survey about how their group is functioning
and make suggestions for improving the group process.
48. Reading Rating Sheets: Students complete a form that rates the effectiveness of the assigned
readings.
49. Assignment Assessments: Students respond to 2 or 3 open‐ended questions about the value of an
assignment to their learning.
50. Exam Evaluations: Students provide feedback about an exam’s learning value and/or format.


Why are you here?
Your progress on the intended workshop learner outcomes.
All your learning, knowledge, and ideas summed up in one place!


Why are you here?
FROM THE BACKGROUND READING
1. a) Define learner outcomes, being sure to capture at least 3 of the 5 main qualities as indicated
by educational researchers.

b) Define formative assessment, being sure to make a comparison to summative assessment
whilst also distinguishing between assessment of and for learning.
TODAY
2. Write a short description (intended for your colleagues who did not come today) of 2 CATs you
plan to try in your course(s) this coming academic term.

a. (name of technique you should really try!)____________________________________
Here is how it works!
b. (name of technique you should really try!)____________________________________
Here is how it works!


Why are you here? (Part 2)
3. Keeping in mind Outcome 2, chose one of the CATs. Re‐write your intended learning outcome to
the descriptions of formative assessment techniques and provide 1 or 2 short sentences about
why you chose these techniques to assess learning of the chosen outcomes. Be sure to include
statements of what you plan to learn about student learning. (You may want to consider an
outcome which you currently have difficulty assessing ). Use the checklist (Exhibit 4.1) to avoid
common problems.
Course name and #_____________. The main goal is:______________________________________
My intended learning outcome :
My reasoning for using ___________________________________ (assessment technique) to
assess the above listed outcome is:
4. Through observation of formative assessment, and reaction to it, in today’s workshop, write
down 2 strategies for how you plan to organize formative assessment in order to make analysis
simple, fast, and informative (e.g., aligning method with cognitive level). Also write 2 strategies
describing how you plan to share feedback with the class (e.g., time span, communication
method, outcome oriented, etc.)
My strategies for organizing formative assessment implementation for simple, fast, and
informative analysis (what must be present so that you are most efficient and learn the most
about student learning):
a.

b.

My Feedback sharing strategies (what must be present for students to learn about their own
learning):
a.

b.
Planning
P hop 2012

Angelo
o: http://www.cttl.ualberta.ca/do
ocuments/WorkkshopI.pdf


Feedback Form
1. Please rate the overall quality of this session on the scale below:

1 2 3 4 5
Very poor Poor Acceptable Good Excellent

2. Please rate the overall usefulness of the session below:

1 2 3 4 5
Useless Not very Somewhat Very Extremely

3. Please rate the effectiveness of the presenter below:

1 2 3 4 5
Not at all Not very Somewhat Very Extremely

4. What did you learn that you can apply to your own teaching? (Please be as specific as possible)

5. How could the session have been more useful to you? (Please be as specific as possible)

6. What was the muddiest point from the entire workshop? (what is still unclear to you?)

Planning Formative Assessment Workshop PDF

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Planning Formative Assessment Workshop PDF

Hochgeladen von

Copyright:

Verfügbare Formate

Planning

Variable Definition Units Value from problem

Das könnte Ihnen auch gefallen