Problem Solving Meeting:

Facilitator’s
Guide
FALL 2012

Ingham Intermediate School District

2630 West Howell Road
Mason, Michigan 48854
517.676.1051
http://www.inghamisd.org
FOREWORD
The purpose of this facilitator’s guide is to provide educators within the Ingham County
service area the tools to run an effective problem solving meeting at the district,
building, and student levels. Included are the meeting worksheets and related resources
for the 4-Step and 8-Step Problem Solving Processes. The 4-Step Problem Solving
Worksheet, (Appendix A) is used for individual or small group academic or behavioral
concerns. The 8-Step Problem Solving Worksheet, (Appendix B) is used for identifying
obstacles within the system that may impede student growth.

Problem solving occurs within the school setting at various levels, and the intensity of
the problem dictates the amount of school resources that are directed toward the
problem solution. The intent of this process is to resolve the problem using a research-
based, systematic, data-driven approach. This guide outlines the necessary steps to
implement the problem solving process.

The Following People Participated in the Development of These

Resources (in alphabetical order):
Eileen McKee, Ph.D.
Johanna Mexico, Ed.S.
Aubrey Tardy, Ed.S.
Michelle Urbanek, Ed.S.
Mary Jo Wegenke, M.A.

Ingham ISD would like to acknowledge the many sources used to create this document,
including the information provided by the problem solving teams of our local districts.
The Meeting Mechanics Module from the Statewide Autism Research and Training
initiative provided valuable tips for holding effective meetings. We would especially like
to thank Dr. George Batsche et al at the University of Florida for sharing their 8-Step
Problem Solving Process.

Problem Solving Meeting: Facilitator’s Guide Page 2

 Table of Contents

Problem Solving Team Introduction pg 4

Role Expectations & Team Members pg 4

4-Step Problem Solving Worksheet pg 6

Step I: Problem Identification pg 6
Gap Analysis pg 6
Gap Statement pg 7
Data Gathering Tools pg 7
RIOT/ICEL Matrix Example pg 10
Step II: Problem Analysis pg 12
Step III: Plan pg 13
Step IV: Do and Review pg 15

8-Step Systems Problem Solving Worksheet pg 18

Step I: Goal Statement pg 18
Step II: Brainstorm pg 19
Step III: Obstacle Selection pg 20
Step IV: Brainstorm pg 21
Steps V & VI: Develop Action & Verification Plans pg 22
Steps VII & VIII Evaluation pg 23

Appendices
Appendix A: 4-Step Problem Solving Worksheet
Appendix B: 8-Step Problem Solving Worksheet
Appendix C: Meeting Norms
Appendix D: RIOT/ICEL Matrix
Appendix E: Alterable Variables Chart
Appendix F: Rate of Improvement (ROI) Chart
Appendix G: Intervention Plan
Appendix H: 4-Step Problem Solving Critical Components Checklist
Appendix I: 8-Step Problem Solving Critical Components Checklist

Problem Solving Meeting: Facilitator’s Guide Page 3

Problem Solving Team Introduction
The purpose of the Problem Solving Team (PST) is to address educational concerns
using current data to determine which research-based strategies and interventions
increase student outcomes. As a team you will:

 Assess major concerns about the student(s) and/or system issues.

 Identify the most important concern to be addressed in the PST meeting.
 Set SMART goals for improvement.
 Design an intervention plan to achieve established goals.
 Develop a plan to measure the response to instruction/intervention.

Role Expectations: Building Level Problem Solving Team Members

Each building must assign staff to support the problem solving model. Primarily,
schools must assign a coordinator who will oversee the problem-solving process and
ensure the integrity and consistency of the model in their building. Principals may
assume this role or should designate individual(s) who will be integral to their problem-
solving team and who can provide guidance and support to the team members. It is
possible that some individuals on the team may fulfill more than one role.

Decision making teams should include parents whenever appropriate. Parents should
be provided documentation of repeated assessment of student’s response to
intervention at regular intervals and be kept informed of changes in intervention
procedures.

Problem Solving Coordinator

Key expectations of the Problem Solving Coordinator:

 Set and notify team members about the days, times and locations of
meetings.
 Contact and inform any guests (specialists, parents) who need to attend
the meeting.
 Ensure that data essential to the decision making process is collected,
including progress monitoring results, intervention plans, fidelity checks,
etc...
 Create the agenda for the PST meeting.

Meeting Facilitator

Key expectations of Meeting Facilitator:

 Facilitate and guide the problem solving process
 Establish and/or remind members of meeting norms (Appendix C)
 Maintain a collaborative atmosphere

Problem Solving Meeting: Facilitator’s Guide Page 4

 Recorder

Key expectations of Recorder:

 Complete 4- or 8-Step Worksheets as the team moves through the
problem solving process.
 Distribute completed problem solving worksheets to team members and
other pertinent individuals as deemed necessary (i.e. parents, building
administrator).

Time Keeper

Key expectations of Time Keeper:

 Monitor meeting time and remind team of agenda time limits.

Problem Solving Meeting: Facilitator’s Guide Page 5

 4-Step Problem Solving Worksheet

Step I: Problem Identification

What is the problem?

Have the team define the primary concern in concrete, descriptive, behavioral terms.
The facilitator may need to ask probing questions to help define the concern in an
observable and measurable way.

EXAMPLE PROBING QUESTIONS:

1. “Johnny is reading below grade level.”
a. What data do you have to support this? (i.e. AIMSweb, DIBELS Next,
Curriculum assessments, etc.)
2. “We are having too many behavior problems during recess.”
a. What data do we have? (i.e. Office Discipline Referrals, SWIS)
b. How many adults are monitoring recess?
c. Who is out on the playground at those times?

EXAMPLE PROBLEM STATEMENT:

1. On an oral reading fluency probe, Johnny reads 12 words correct per minute with
7 errors.
2. During 3rd-5th grade recess, an average of 3 students receives an Office
Discipline Referral on a daily basis.

Academic and/or Behavior GAP Analysis

These questions will help to determine how discrepant the performance is from the
expectation, and will help to facilitate goals. These questions help frame the context and
severity of the problem.

What is the expected/benchmark level of performance for building level/grade

level/student level?
 Refer to standardized norms; establish local norms if needed.
 Consider what is developmentally appropriate.

What is the peer level of performance at the building level/grade level/classroom

/student level?
 Refer to local norms (AIMSweb, DIBELS Next, Classroom data, SWIS).

Problem Solving Meeting: Facilitator’s Guide Page 6

What is the peer subgroup level of performance?
 Subgroup= ethnicity, SES, special education, gender
 Refer to local norms (AIMSweb, DIBELS Next, Classroom data, SWIS).
 Refer to classroom data (i.e., Test scores, Running Record).

GAP Statement

What is the discrepancy between current performance and expected

performance?

EXAMPLE GAP STATEMENT:

1. On an oral reading fluency probe, Johnny reads 12 words correct per minute with
7 errors, while the typical third grader (using AIMSweb national 50 th percentile)
reads 87 words read correct per minute.
2. We expect 7th graders to receive one or fewer office discipline referrals (ODR)
per year (using SWIS norms) during transition times, and our data shows the
average 7th grader is receiving 2 per month.

Data Gathering Tools

The RIOT/ICEL Matrix is an organizing framework that increases schools’ confidence
both in the quality of the data that they collect and the findings that emerge from the
data. The top horizontal row of the RIOT/ICEL table includes four potential sources of
student information: Review, Interview, Observation and Test (RIOT). The leftmost
vertical column of the table includes four key domains of learning to be assessed:
Instruction, Curriculum, Environment and Learner (ICEL). A common mistake is to
underestimate the degree to which instructional strategies, curriculum demands, and
environmental influences impact academic performance.

RIOT
1) Review of historical records and products
2) Interview of teachers, students and parents
3) Observe student performance in real time functional settings
4) Test student through careful use of appropriately matched measurement
technologies

Problem Solving Meeting: Facilitator’s Guide Page 7

ICEL
INSTRUCTION: Instruction is how curriculum is taught. Here are some
example variables in this area:
Individual
 Instructional decision making
regarding placement of the student
in groups
 Use of progress monitoring
 Communication of expectations and
criteria for success
 Differentiated Instruction
 Direct instruction with explanations
and cues
 Use of a variety of practice and
application activities
 Pace and presentation of new
content
CURRICULUM: Curriculum refers to what is taught. Here are some example
variables in this area:
Individual
 Accommodations
 Supplementary instruction
 Interventions
 Access to instruction (time,
attention, behavior, attendance)
 Instructional materials
 Arrangement of the
content/instruction
Problem Solving Meeting: Facilitator’s Guide
Group/System
 Instructional decision making
regarding selection and use of
materials
 Use of progress monitoring
 Explicit Instruction
 Differentiated Instruction
 Sequencing of lesson designs to
promote success
 Use of a variety of practice and
application activities
 Pace and presentation of new
content
 Block of time allotted per subject
Group/System
 Presence of Core Curriculum
 Universal behavior
expectations/PBIS
 Staff training in curriculum
 Percentage of students at
benchmark/meeting grade level
expectations
 Long-range direction for instruction
 Alignment to standards
 Instructional philosophy/approaches
 Instructional materials
 Stated outcomes for the course of
study
Page 8
 ENVIRONMENT: The environment is where the instruction takes place. Here
are example variables in this area:

Individual/Group/System
 Physical arrangement of the  Expectations
classroom or other problem location  Peer context
 Furniture/equipment  Peer and family influence
 Rules  Task pressure
 Management plans  Adult supervision
 Routines

LEARNER: The learner is who is being taught. This is the last domain that is
considered and is only addressed when the curriculum and
instruction are found to be appropriate and the environment is
accommodating. Here are some example variables in this area:

Individual
 Student’s current knowledge, or ‘prior knowledge’
 Academic performance data
 Social/behavioral performance data
 Student’s skills and motivation
 Curriculum and instruction are appropriate
 Student’s ‘ability’, race, gender or family history

Problem Solving Meeting: Facilitator’s Guide Page 9

 EXAMPLE OF A COMPLETED ICEL/RIOT MATRIX:
(Completed for Sarah, a 2nd grade student weak in math)

Review Interview Observe Test

Instruction Has not been Has worked in small Explicit, Fidelity checks show
retained. groups with classroom differentiated teacher is using the
teacher (re-teaching instruction curriculum materials
No progress lessons). observed. as designed.
monitoring data in
this area. Teacher confirms 60-
minute math block occurs
daily.
Curriculum A research-based Teacher reports no Curriculum Unit tests are used
core curriculum is additional interventions/ implemented with to assess learning
in place and accommodations have fidelity. (class average =
aligned with been tried 80%).
common core
standards.
Environment n/a Classroom management Classroom n/a
plan in place. expectations &
rules posted.
Very few interruptions to
instruction. Student is seated
up front and near
teacher.

Students are on-

task and engaged.

Transition times are

minimal.
Learner Past math grades Parents, past teachers, Sarah is engaged AIMSweb M-COMP
and report card and Sarah report in the classroom benchmark data.
comments difficulties with math. except during math
indicate below lessons. Sarah averages
grade level Teacher reports student 45% on unit tests.
performance. requires manipulatives to Sarah uses fingers
accurately solve math & manipulatives
Attendance problems.
records (good
attendance).

Passed all vision

& hearing
screenings.

Problem Solving Meeting: Facilitator’s Guide Page 10

After completing the RIOT/ICEL chart, review your GAP statement to facilitate
discussion around the questions below.

EXAMPLE GAP STATEMENT:

1. Sarah scores 14 points on the AIMSweb Winter M-COMP, while 2nd grade peers
score 28 points.

Is this a system or curriculum issue? (If yes, begin 8 Step Problem Solving Process)

EXAMPLE QUESTIONS TO HELP DETERMINE IF SYSTEM ISSUE:

 How many (number/percentage) students are performing below expectations?
 What research-based curriculum is being taught?
 Is the curriculum being implemented with fidelity?
 What supplemental supports are provided in our school?

Is this a skill or performance deficit?

Is the student ABLE to perform the skill in question?
 If yes, when is the skill performed versus not performed? (performance deficit)
 If no, assume it’s a skill deficit.
 Not sure??? Collect more data.
Do we need more data?
 If yes, continue/discontinue with problem solving meeting?
 Continue- meeting can continue although more data is needed
 Discontinue- more data is needed to move ahead

*** Facilitator’s Note: In some instances, the current data may be sufficient to continue
the problem solving meeting, despite the need for more data. This may be collected as
part of the intervention plan and included in the follow-up review. For example, you may
need to collect data to determine if the problem is a skill or performance deficit.

 If collecting more data, what additional data do we need?

o What skill(s) do we want to assess?
o What assessment tools are available?
o Do we have data to determine other students’/grade level’s performance?
o Who will collect this data?
o When will we collect this data?

 When will we meet again?

o Schedule the next meeting date and time as a team, providing adequate
time for additional data collection.

Problem Solving Meeting: Facilitator’s Guide Page 11

 STEP II: Problem Analysis

These questions will help the team identify the main concerns, reasons the problem
exists, form hypotheses, and develop a goal statement for replacement behaviors.

Hypotheses and Prediction Statement Development- Main Problem(s)

What are the most likely reasons this problem is occurring?

 Refer back to the RIOT/ICEL Matrix
 Address potential domains of instruction, curriculum, environment, and learner.

What changes could be made to: The instruction? The curriculum? The environment?

EXAMPLE QUESTIONS:

 Do we need more instructional time?

 Could we improve the curriculum implementation integrity?
 Are behavior expectations posted?
 Do we need to re-teach behavior expectations?
 What skills are we most concerned with our students learning?
 Based on the data, what is our greatest area of need?
 Refer to the Alterable Variables Chart (Appendix E)

EXAMPLE HYPOTHESES/PREDICTION STATEMENTS:

1. If the district implemented a reading curriculum with fidelity, then students’

reading fluency performance would increase.
2. A positive reinforcement intervention for James targeting raising his hand before
speaking will reduce his blurting during classroom instruction.

Problem Solving Meeting: Facilitator’s Guide Page 12

 STEP III: Plan

This section creates an implementation plan to address the selected goal(s). See
Appendix G for a blank intervention plan chart.

Goal Statement
Identify the desired behavior/skill and determine, based on the data, a reasonable goal
within the timeline the team establishes. A SMART goal is written to be Specific,
Measureable, Attainable, Realistic, and Timely.

EXAMPLE SMART GOAL STATEMENT:

1. By the target date of April 21, 2012 the student will read 43 correct letter sounds
and 10 whole words on DIBELS Next Nonsense Word Fluency probes.
 In establishing individual student academic goals refer to the Rate of
Improvement Chart in Appendix F or local norms.
2. By the target date of November 10, 2012 the student will achieve a weekly
average of 80% of Check-In/Check-Out points.
 In establishing behavioral goals, refer to SWIS national averages or local
behavioral data.
3. By the target date of January 20, 2012 the Kindergarten teachers will be trained
and fully implementing K-PALS in their classrooms.

Intervention
Name: Select a research-based intervention or strategy that targets the concern
identified in your Goal Statement.

Group Size: Identify the group size and check the corresponding box.

Number of weeks: Establish the number of weeks before the next meeting and data
review (4-6 data points needed).

Implementation
Who: Decide who will implement the intervention.

Number of times per week: Establish the number of intervention sessions per week that
will take place.

Number of minutes per day: Determine how long each session will last.

Problem Solving Meeting: Facilitator’s Guide Page 13

Fidelity
Who: Determine who will monitor implementation fidelity.

When: Establish when and how often fidelity checks will be completed.

What: Select a fidelity measurement tool/checklist to utilize.

Feedback: Determine when and how feedback will be provided to the individual
implementing the selected intervention.

Progress Monitoring
Who: Determine who will monitor progress.

Tool: Select a measurement tool that assesses the skill(s) targeted by the
intervention(s). Discuss as a team where the data will be recorded.

How often: Establish how often data will be collected (i.e., weekly, bi-weekly).

Next Meeting

Schedule a follow-up meeting date that corresponds with the number of weeks the
intervention will take place and will allow time for the collection of data utilizing the
selected progress monitoring tool.

Problem Solving Meeting: Facilitator’s Guide Page 14

 STEP IV: Do & Review

Response to Instruction/Intervention: Is the Plan Working?

Attach data for each review date. Include progress monitoring data, fidelity data, and
any other relevant data that was collected since the last meeting.

Are interventions being implemented as designed?  Yes  No

Fidelity of implementation was checked ______ times by ____________________.

 Refer to your implementation fidelity data.

The response to instruction/intervention is:

 Positive Inconsistent Not Responding

Positive: Rate of Improvement/Growth Rate is calculated at 1.5 times the rate of

improvement.
 Student’s current Rate of Increase/Growth Rate is calculated and indicates
the performance gap is being closed.
 Trendline is with or above the Aimline, indicating the performance gap is
being closed.
 Student is achieving at or near the behavioral goal(s) established in the goal
statement/plan.
Inconsistent:
 Performance varies substantially.
 At times, student is achieving goal and at other times they are performing
below expectations/Aimline.
Not Responding:
 Trendline below the Aimline.
 Student is not making progress toward behavioral or academic goal.

If positive:
 Continue current instructional supports (Change the number of weeks and
establish the next meeting date on the Step III Plan chart)
 Adjust goal upward (Record new goal on Step III Plan chart)
 Fade supports (Record on Step III Plan chart)

Problem Solving Meeting: Facilitator’s Guide Page 15

If inconsistent or not responding:
 Recheck hypothesis (Step II)
 Did the hypothesis accurately identify the problem and the reason(s) it
was occurring?
 Was the intervention/instruction implemented as designed?  Yes  No
 Refer to your implementation fidelity data.

 If yes:

 Was intervention aligned with the hypothesis?  Yes  No

 If no, discuss alternate intervention(s) that align with the hypothesis.

 Should the intensity be increased?  Yes  No

 Intensity can be increased by providing more time, creating smaller
groups, providing intervention one-on-one, or by increasing the number
of intervention sessions per week.

 Are there other alterable variables to be considered  Yes  No

with this hypothesis and intervention?
 Refer to Alterable Variables Chart in Appendix E and list considered
variables in Step IV.

 Are there other hypotheses to consider?  Yes  No

 If yes, review and address alternate hypotheses from Step II.
Determine if intervention(s) should change to target alternate
hypotheses. Record the changes and proceed from Step II.

 If no (Refer to fidelity implementation data):

 Was it the frequency of implementation?  Yes  No

If yes, how many sessions were held? ___ out of ___ possible
Please explain the reason(s): ________________________________
________________________________________________________

 Was it the accuracy of implementation?  Yes  No

If yes, please explain the reason(s): ___________________________
________________________________________________________

Problem Solving Meeting: Facilitator’s Guide Page 16

 8-Step Problem Solving Process
4-Step ID’s System Issue

1. Develop Goal 2. Brainstorm 3. Select Obstacle

Statement Obstacles & to Address
Resources

6. Verify 5. Create Plan to 4. Brainstorm

Completion Reduce Obstacle Strategies to
of Action _____ will do _____ Reduce the
_____ will do _____ Obstacle

7. Evaluate Plan 8. Did it Improve Outcome

(obstacle reduced?) Stated in Step 1?

NO - NO –
Go back to steps Go back to step 3
YES – Celebrate
5 & 6 to adjust & select alternate
YES! briefly then select obstacle
action plan
Yippee next obstacle, if
needed
 8-Step Systems Problem Solving Worksheet

Once a system issue has been identified using the 4-Step Problem Solving Process,
begin the 8-Step Problem Solving Process. If the 4 Step Problem Solving Process has
not been completed it may be difficult to easily identify a problem hypothesis. In that
instance it is recommended that you complete the first two pages of the 4-Step
Worksheet to develop your hypothesis, then transition to the 8-Step Worksheet. Begin
the 8-Step Worksheet by stating or transferring the problem where indicated.

EXAMPLE PROBLEM HYPOTHESIS:

We have too many Office Discipline Referrals before and after school at the bus loop
due to inadequate supervision.

STEP I: Goal Statement

Create SMART Goal:

(A SMART goal is written to be Specific, Measureable, Attainable, Realistic, and

Timely)

EXAMPLE SMART GOAL:

We will decrease the number of Office Discipline Referrals occurring at the bus loop
before and after school by 50% by increasing the adult supervision from 2 to 6
adults.
 STEP II: Brainstorm

Brainstorm all available resources that might facilitate achievement of desired outcome
and all obstacles that might prevent achieving the desired outcome.
During this step it is imperative that an open, collegial atmosphere is established.
Members of the team should feel that they have an equal voice, regardless of their
professional roles. This is not the time to determine the feasibility of the resources; just
to list as many as you can. All ideas are good ideas at this point. Participants should
state their ideas quickly, without taking the time to explain, defend or convince the other
members of the problem solving team.
It is recommended that the facilitator outline the specific process for brainstorming that
will be used. An example might be to give the team time to jot down their ideas, then go
around the table and have each person quickly list one of their thoughts. Each person
has the option of saying, “pass” to skip their turn. The conversation continues in this
round-robin fashion until each member of the team has “passed”.
*** Facilitator’s Note: If vague obstacles are provided, such as lack of time or money,
the facilitator should ask probing questions to elicit more specific answers. For example,
if lack of time is listed as an obstacle, the facilitator may ask “How would we use more
time to achieve our SMART goal?” or “Exactly how much time would be needed?”

EXAMPLE BRAINSTORM FOR GOAL IDENTIFIED IN STEP I:

(We will reduce the number of ODR’s occurring at the bus loop before and after school
by 50% by increasing the adult supervision from 2 to 6 adults)
Resources (+) Obstacles (-)
Staff members Adult Availability
School-wide behavior expectations Contract language
PBIS lesson plans Staff buy-in & resistance
Scheduling
Training on end of day/bus procedures
Administrative support

Problem Solving Meeting: Facilitator’s Guide Page 19

 STEP III: Obstacle Selection

In this step the team will discuss the obstacles identified in Step II and prioritize which
one needs to be addressed first so the problem can be solved. The facilitator should
actively lead this process by:

1. Clarifying any ambiguity in the obstacles listed.

2. Grouping similar ideas
3. Helping the team prioritize the primary obstacle

***Facilitator’s Note: If the group is having difficulty agreeing on the primary obstacle
you may need to employ a specific process such as having each member place a
sticker next to the option they feel is most important; thereby giving a visual
representation of the breakdown for further discussion and consensus building.

It is important to focus on one obstacle at a time, despite the presence of several that
may need to be targeted.

EXAMPLE PRIMARY OBSTACLE CHOSEN:

Lack of Staff Buy-in

Problem Solving Meeting: Facilitator’s Guide Page 20

 STEP IV: Brainstorm

Reduction/Elimination of Obstacle

Brainstorm strategies to reduce or eliminate the obstacle identified in Step III and record
them below. Follow the same format for brainstorming identified in Step II. Remember
these are only ideas. Do not consider feasibility or implementation at this stage.

EXAMPLE BRAINSTORM FOR PRIMARY OBSTACLE OF LACK OF STAFF BUY-IN:

 Show staff current ODR data occurring at the bus loop

 Offer incentive to staff for being present during bus pick-up/drop-off
 Review & adjust staff schedules to increase availability
 Initiate discipline procedures for resistant staff

***Facilitator’s Note: Do not allow team members to make negative comments about
the ideas generated. Remind them this is not the time to evaluate the feasibility of the
idea.

Problem Solving Meeting: Facilitator’s Guide Page 21

 STEPS V & VI: Develop Action & Verification Plans

Using the list generated in Step IV as a starting point, discuss the feasibility and impact
each item may have on eliminating the obstacle identified in Step III. Include those
items considered to be “low hanging fruit” which are easiest to implement immediately.
Create an action plan which specifies who will do what and by when.

EXAMPLE PLAN TO REDUCE THE OBSTACLE OF LACK OF STAFF BUY-IN:

Step 5 Action Plan Step 6 Completion Verification

Who Action By When Who When Completed?
School Social The SSW will share the ODR At the Principal At the □ Yes
Worker data from the last 3 months Oct. staff Oct. staff □ No
for before and after school. meeting meeting Verified by:

School Create and present incentive Oct. staff Principal At the □ Yes
Improvement plan to staff. meeting Oct. staff □ No
Team meeting Verified by:

Principal & Review staff schedules and To be Principal At the □ Yes

School identify staff with availability presented Oct. staff □ No
Improvement at the beginning and end of at the meeting Verified by:
Team the school day. Oct. staff
meeting

Problem Solving Meeting: Facilitator’s Guide Page 22

 STEPS VII & VIII: Evaluation

What is the plan to evaluate the reduction/elimination of the obstacle identified in

Step III?

EXAMPLE PLAN TO EVALUATE ELIMINATION OF LACK OF STAFF BUY-IN:

After discussing the ODR data and problem of lack of supervision at the bus loop
with the staff, the team will implement the incentive plan with school staff with
availability at the beginning and end of the day. The principal or School Improvement
Team member will record the number of staff members present at the bus loop
before and after school 3 times per week during the month of November. The
obstacle will be considered eliminated if the average number of adults at the bus
loop increases to 6 adults.

*** Facilitator’s Note: If the data indicates that the obstacle has not been
reduced/eliminated, then return to Steps V and VI to adjust the Action Plan.

What is the plan for evaluating progress toward achieving the desired outcome
specified in the SMART goal in Step I?

EXAMPLE PLAN TO EVALUATE THE REDUCTION OF ODR’S AT THE BUS LOOP

BY INCREASING ADULT SUPERVISION:

The School Social Worker will share the report on ODR’s occurring at the bus loop
before and after school with the entire staff at the December staff meeting. A
reduction of 50% or more in ODR’s occurring at the bus loop will indicate successful
achievement of our SMART goal.

*** Facilitator’s Note: If data does not show improvement or progress toward the
outcome specified in Step I, then you may need to return to Step III, select an alternate
obstacle, and continue the process. If data does show progress toward the outcome,
celebrate! Return to Step III if needed to address additional obstacles.

Problem Solving Meeting: Facilitator’s Guide Page 23

APPENDIX A
 4-Step Problem Solving
Worksheet

Follow Up Date _____________

District ___________ Building ____________ Initial Date________Follow Up Date _____________
Follow Up Date _____________
Grade Level _______ Student ______________________________

Step I: Problem Identification

What is the problem?

Academic and/or Behavior GAP Analysis

What is the expected/benchmark level of performance for building level/grade level/student level?

What is the peer level of performance at the building level/grade level/student level?

What is the peer subgroup/level of performance?

GAP Statement
What is the discrepancy between current performance and expected performance?
 Data Gathering Tools

RIOT/ICEL MATRIX
Review Interview Observe Test
Instruction

Curriculum

Environment

Learner

Is this a system or curriculum issue?  Yes (Begin 8- Step Problem-Solving Process)

 No

Is this a skill or performance deficit?  SKILL  PERFORMANCE

Do we need more data?

If yes, continue or discontinue with problem solving meeting?  Continue Discontinue

If discontinuing: What additional data do we need? ___________________________________

When will we meet again? __________________________________________

Step II: Problem Analysis

Hypotheses and Prediction Statements Developed-Main Problem(s)

1.

2.

3.
 Step III: Plan
Next
Intervention Implementation Fidelity Progress Monitoring
Meeting
Mins/Day

Feedback
Schedule
# Weeks

Times/
Group
Name

When
Week

Often
What
Who

Who

Who

How
Tool
Size

GOAL: By the target date of the student will

1
 2-4
 5-8

Whole
Class
GOAL: By the target date of the student will
1
 2-4
 5-8

Whole
Class
GOAL: By the target date of the student will
1
 2-4
 5-8

Whole
Class
GOAL: By the target date of the student will
1
 2-4
 5-8

Whole
Class
 Step IV: Do & Review

Response to Instruction/Intervention: Is the Plan Working?

Attach data for each review date:

Review Date: ______________________________

Are interventions being implemented as designed:  Yes  No

Fidelity of implementation was checked ___________ times by ______________________________.

The response to instruction/intervention is:  Positive  Inconsistent  Not Responding

1. If positive:
 Continue current instructional supports
 Adjust goal upward
 Fade supports

2. If inconsistent or not responding:

 Recheck hypothesis (Step II)
 Was the intervention/instruction implemented as designed?  Yes No

 If yes:
 Was intervention aligned with the hypothesis?  Yes No
 Should the intensity be increased?  Yes No
 Are there other alterable variables to be  Yes No
considered with this hypothesis and intervention?
 Are there other hypotheses to consider?  Yes No

 If no:
 Was it the frequency of implementation?  Yes No
If yes, how many sessions were held? _____ out of _____ possible
Please explain the reason(s): __________________________________
 Was it the accuracy of implementation?  Yes No
If yes, please explain the reason(s): _____________________________
APPENDIX B
 8-Step Systems
Problem Solving Worksheet

District ___________________________________ Initial Date __________________________

Building __________________________________ Follow Up Date ______________________

Grade Level(s) ____________________________ Follow Up Date ______________________

Problem Hypothesis: ________________________________________________________________

Step I: Goal Statement

SMART Goal:

___________________________________________________________________________________

___________________________________________________________________________________

Step II: Brainstorm

Resources (+) Obstacles (-)

Step III: Obstacle Selection

Primary Obstacle:
 Step IV: Brainstorm

Reduction/Elimination of primary Obstacle:

Steps V & VI: Develop Action &

Verification Plans

Action Plan Completion Verification

Who Action By When Who When Completed?

□ Yes □ No
Verified by:

□ Yes □ No
Verified by:

□ Yes □ No
Verified by:

□ Yes □ No
Verified by:

Steps VII & VIII: Evaluation

What is the plan to evaluate the reduction/elimination of the obstacle identified in Step III?

What is the plan for evaluating progress toward achieving the desired outcome specified in the
SMART goal in Step I?
APPENDIX C
 Sample Meeting Norms
 We will focus only on those things over which we have control.
 We will commit to always seek better ways to teach such that more students will learn at
 high levels.
 We will use data rather than opinions and feelings to make decisions and hold ourselves
 accountable.
 Begin and end on time.
 We will use an agenda to guide us through the process.
 We will come prepared with all data and materials to be actively involved in the process.
 Limit side bar conversations.
 Begin on time.
 End on time.
 Attend all meetings.
 Respect questions.
 Monitor your own airtime.
 Stay on agenda.
 Do assignments prior to meetings.
 Hold yourself personally accountable.
 Do not use hostile language.
 Respect the group.
 Listen attentively.
 Attend to goals and objectives.
 Listen respectfully.
 Discuss issues, not people.
 Probe ideas; do not criticize people.
 Show respect for views of others.
 Avoid side conversations.
 Assume positive intentions.
 Observe basic conversational courtesies.
 Avoid use of cell phone and checking personal emails.
APPENDIX D
RIOT/ICEL MATRIX

Review Interview Observe Test

Instruction

Curriculum

Environment

Learner
APPENDIX E
 Alterable Variables Chart
Alterable Components Specific Adjustments

Opportunities to Learn Increase attendance Provide instruction Increase Vary schedule of Add another
(Time/Concentration of daily opportunities to easy/hard instructional period
Instruction) respond tasks/skills (double dose)

Program Efficacy Preteach Use extensions of Supplement core Replace current core Implement specially
components of core the core program with appropriate program designed program
program materials

Program Implementation Model lesson Monitor Provide coaching Provide additional Vary program/lesson
delivery implementation and ongoing support staff development schedule
frequently

Grouping for Instruction Check group Reduce group size Increase teacher-led Provide individual Change instructor
placement instruction instruction

Coordination of Instruction Clarify instructional Establish concurrent Provide Establish Meet frequently to
priorities reading periods complementary communication examine progress
reading instruction across instructors
across periods
APPENDIX F
 AIMSWEB WEEKLY 50TH
PERCENTILE RATES OF IMPROVEMENT:

Early Literacy Measures:

Grade Levels
st nd rd
Kdg 1 2 3 4th 5th 6th 7th 8th 9th-12th
LNF: 0.86 0.44

LSF: 0.94 0.61

PSF: 1.11 0.39 0.17

NWF: 0.83 0.94 0.56

NNF=Letter Naming Fluency, LSF=Letter Sound Fluency, PSF=Phoneme Segmentation Fluency,

NWF=Nonsense Word Fluency

Reading Measures:
Grade Levels
st nd rd
Kdg 1 2 3 4th 5th 6th 7th 8th 9th-12th
R-CBM: 1.50 1.22 1.11 0.89 0.89 0.69 0.64 0.47 0.47

MAZE: 0.08 0.17 0.28 0.17 0.19 0.08 0.08

R-CBM=Oral Reading Fluency, MAZE-Comprehension

Early Numeracy Measures:

Grade Levels
st nd rd
Kdg 1 2 3 4th 5th 6th 7th 8th 9th-12th
OCM: 1.08 0.61

NIM: 0.64 0.64

QDM: 0.47 0.39

MNM: 0.31 0.25

OCM=Oral Counting Measure, NIM=Number Identification Measure, QDM=Quantity Discription

Measure, MNM=Missing Number Measure
Math Measures:
Grade Levels
st nd rd
Kdg 1 2 3 4th 5th 6th 7th 8th 9th-12th
M-COMP: 0.83 0.67 0.94 0.92 0.50 0.42 0.36 0.25

M-CAP: 0.39 0.22 0.17 0.08 0.17 0.22 0.08 0.08

M-COMP=Math Computation, M-CAP=Math Concepts and Applications

Written Expression Measures:

Grade Levels
st nd rd
Kdg 1 2 3 4th 5th 6th 7th 8th 9th-12th
TWW: 0.39 0.47 0.36 0.28 0.28 0.36 0.22 0.17 0.17

CWS: 0.25 0.33 0.33 0.38 0.33 0.44 0.17 0.19 0.19

TWW=Total Words Written, CWS=Correct Writing Sequences

APPENDIX G
 Step III: Plan

Intervention Implementation Fidelity Progress Monitoring

Name Group # Who Times/ Min/ Who When What Feedback Who Tool How
Size Weeks Week Day Schedule Often
th
GOAL: By the target date of 2/21/13 the student will read 105 wcm on a DIBELS Next 5 grade ORF probe.
Read Naturally X 1 6 Interventionist 3x 45 RTI Every Read Naturally Fidelity Every Interventionist DIBELS Next ORF, 5th 2/24/13
2-4 Coach Friday Checklist Monday grade @
5-8 8:30
a.m.
Whole
Class

GOAL: By the target date of the student will

6 Minute Solution 1 6 Interventionist 3x 10 RTI Every 6 Minute Solution Every
X 2-4 Coach Tuesday Fidelity Checklist Wednesday
5-8

Whole
Class

GOAL: By the target date of the student will

1
2-4
5-8

Whole
Class

GOAL: By the target date of the student will

1
2-4
5-8

Whole
Class
APPENDIX H
 4-STEP PROBLEM SOLVING MEETING
CRITICAL COMPONENTS CHECKLIST
Directions: This checklist can be used to evaluate your problem solving team effectiveness on an
ongoing basis. It is recommended to collect baseline data and periodically review the checklist as a team
to improve the process. Please use the scale provided to indicate the degree to which each critical
component was addressed as part of the problem solving process. See the attached rubric to rate the
degree to which each component is present.

Component 1 = Present Evidence/Comments

2 = Partially Present
3 = Absent
Steps I & II: Problem Identification & Analysis
1. Problem was described in observable and
measureable terms. 1 2 3

2. Gap analysis was conducted and a gap

statement was developed. 1 2 3

3. Hypotheses were developed across multiple

domains. 1 2 3

4. Hypotheses were developed to determine if

the student was not performing because of a
1 2 3
performance and/or skill deficit.

5. Data were used to determine hypotheses for

why the behavior was not occurring. 1 2 3

Step III: Intervention Development and Implementation

6. Short- and long-term goals were clearly
stated in relation to the benchmarks. 1 2 3

7. Interventions were developed to address

concerns identified by hypotheses. 1 2 3

8. An intervention plan specifying the logistics

of the intervention(s) selected was
1 2 3
developed.

Step IV: Program Evaluation

9. Data were provided demonstrating that the
intervention plan was implemented with
1 2 3
integrity.

10. Progress-monitoring data were

collected/scheduled. 1 2 3

11. Criteria for determining a positive response

to instruction/intervention was defined. 1 2 3

12. A plan for continuing, adjusting, or fading the

intervention plan was developed. 1 2 3
 CRITICAL COMPONENTS CHECKLIST RUBRIC

1. Problem was described in observable and measureable terms

 Present = The problem was described in observable and measureable terms using available data.
 Partially Present = The problem was described using available data, but was not observable
AND/OR measurable.
 Absent = The problem was not described in observable or measureable terms and was not based
on available data.

2. Gap analysis conducted and goal statement developed

 Present = Data were used to calculate the Performance gap compared to peers and benchmarks.
A goal statement was also developed.
 Partially Present = One or more components listed in “Present” was absent (i.e. comparison to
peers, comparison to benchmarks, goal statement)
 Absent = No analysis was conducted to determine the performance gap and no goal statement
was written.

3. Hypotheses were developed across multiple domains

 Present = Reasons for the student not performing were developed. The reasons provided span
multiple hypothesis domains (e.g., child, curriculum, peers, family/community, classroom,
teacher).
 Partially Present = Reasons for the student not performing were developed, but the reasons do
not span multiple hypothesis domains (e.g., curriculum hypotheses only).
 Absent = Reasons for the student not performing were not developed.

4. Hypotheses developed to determine performance or skill deficit

 Present = Hypotheses were developed to determine whether the student was not performing was
due to a performance and/or skill deficit.
 Partially Present = A discussion of performance versus skill deficits was provided, but no formal
hypotheses were developed addressing whether the student was not performing because of a
performance and/or skill deficit.
 Absent = No discussion of, or hypotheses investigating, whether the student was not performing
because of a performance and/or skill deficit took place.

5. Data collected to determine hypotheses

 Present = Data were collected using RIOT (Review, Interview, Observe, Test) procedures for all
hypotheses to determine the reasons that are likely to be barriers to student performance.
 Partially Present = Data were collected using RIOT (Review, Interview, Observe, Test) procedures
for some hypotheses to determine the reasons that are likely to be barriers to student
performance.
 Absent = Data to determine the reasons that are likely to be barriers to student performance were
not collected.

Adapted from the Florida Problem-Solving/Response to Intervention Project (2008) August 2012.
6. Short- and long-term goals clearly stated
 Present = Short- and long-term goals for student RTI were clearly stated in relation to the
benchmark.
 Partially Present = Short- and long-term goals for student RTI were clearly stated but did not relate
to the benchmark.
 Absent = Short-term and long-term goals were not stated.

7. Interventions linked to barriers from verified hypotheses

 Present = The components of the intervention plan can be linked directly to concerns identified by
hypotheses.
 Partially Present = Some of the components of the intervention plan can be linked directly to
concerns identified by hypotheses.
 Absent = Few or none of the components of the intervention plan can be linked directly to
concerns identified by hypotheses.

8. Intervention plan was developed specifying logistics

 Present = An intervention plan was developed specifying all logistical information (i.e., who, what,
when, where, materials needed)
 Partially Present = An intervention plan was developed, but some logistical information (i.e., who,
what, when, where, materials needed) was missing.
 Absent = No intervention plan specifying logistical information was developed.

9. Data were provided demonstrating intervention integrity

 Present = Quantifiable data were provided demonstrating that the intervention plan was
implemented as intended.
 Partially Present = Information was provided to support claims that the intervention plan was
implemented as intended, but no quantifiable data were present.
 Absent = No information on the degree to which the intervention plan was implemented as
intended was provided.

10. Progress-monitoring data was collected/scheduled

 Present = Progress-monitoring data was collected at an appropriate frequency using measures
that are sensitive to small changes in the target skill.
 Partially Present = Progress-monitoring data was not collected frequently enough AND/OR was
collected using measures that were not sensitive to small changes in the target skill.
 Absent = Little or no progress-monitoring data was collected.

11. Criteria for determining a positive response to instruction/intervention was defined

 Present = Current data (i.e., local norms, ROI, SWIS national average) was used to determine
what constitutes a positive response to instruction/intervention.
 Partially Present = Estimates were used to determine what constitutes a positive response to
instruction/intervention.
 Absent = No data was used to determine what constitutes a positive response to
instruction/intervention.
Adapted from the Florida Problem-Solving/Response to Intervention Project (2008) August 2012.
12. Plan for continuing, adjusting, or fading the intervention plan was developed
 Present = A plan for continuing, adjusting, or fading the intervention plan was developed based on
the response to instruction/intervention
 Partially Present = A plan for continuing, adjusting, or fading the intervention plan was provided,
but it did not link directly to the response to instruction/intervention.
 Absent = No plan for continuing, adjusting, or fading the intervention plan was developed.

Adapted from the Florida Problem-Solving/Response to Intervention Project (2008) August 2012.
APPENDIX I
 8-STEP PROBLEM SOLVING MEETING
CRITICAL COMPONENTS CHECKLIST
Directions: This checklist can be used to evaluate your problem solving team effectiveness on an
ongoing basis. It is recommended to collect baseline data and periodically review the checklist as a team
to improve the process. Please use the scale provided to indicate the degree to which each critical
component was addressed as part of the problem solving process. See the attached rubric to rate the
degree to which each component is present.

Component 1 = Present Evidence/Comments

2 = Partially Present
3 = Absent
1. SMART goal is written based on available
data.
1 2 3

2. Resources and Obstacles were discussed

during brainstorming.
1 2 3

3. Consensus was established to determine the

primary obstacle.
1 2 3

4. Resources were considered when

brainstorming to reduce/eliminate the
selected obstacle. 1 2 3

5. District or School-wide Action and

Verification Plans specifying the logistics of
implementation was developed. 1 2 3

6. A plan was developed to evaluate the

reduction/elimination of the primary obstacle.
1 2 3

7. A plan was developed to reconvene and

evaluate progress.
1 2 3
 CRITICAL COMPONENTS CHECKLIST RUBRIC

1. SMART goal written based on available data (i.e. data from the 4 Step process, SWIS, DIBELS,
AIMSweb, local norms)
 Present= Data was utilized and a SMART goal was written.
 Partially Present= A SMART goal was written, but was not based on measureable and observable
data.
 Absent= Neither of the above conditions were met.

2. Resources and Obstacles were discussed during brainstorming

 Present= Both resources and obstacles were listed AND each team member had an opportunity to
contribute.
 Partially Present= Resources OR obstacles were discussed AND/OR not all team members had
the opportunity to contribute.
 Absent= The will of the group was not heard.

3. Consensus was established to determine the primary obstacle

 Present= Team members established consensus to select an obstacle.
 Partially Present= The team agreed to move on without establishing consensus.
 Absent= The will of the group was not heard AND/OR one individual selected the obstacle.

4. Resources were considered when brainstorming to reduce/eliminate obstacle

 Present= Multiple resources were listed to reduce/eliminate the obstacle.
 Partially Present= Few resources were discussed.
 Absent= Resources were not discussed.

5. District or School-wide Action and Verification Plans specifying the logistics of implementation were
developed
 Present= Action and Verification Plans were developed specifying all logistical information (i.e.,
who, what, when).
 Partially Present= Plans were developed, but some logistical information (i.e., who, what, when)
was missing.
 Absent= No plans specifying logistical information were developed.

6. A plan was developed to evaluate the reduction/elimination of the primary obstacle

 Present= A plan was developed and the team reconvened to evaluate progress using measurable
data.
 Partially Present= A plan was developed and the team evaluated progress, but it was not based
on measurable data. OR the team did not reconvene.
 Absent= No plan was developed.
7. A plan was developed to reconvene and evaluate progress toward the SMART goal
 Present= The team established a plan to reconvene and evaluate progress toward the SMART
goal using measurable data (i.e. data from the 4 Step process, SWIS, DIBELS, AIMSweb, local
norms).
 Partially Present= The team discussed reconvening to evaluate progress, but a clear plan was not
developed.
 Absent= A plan to reconvene and evaluate progress was not developed.