Professional Development Project Evaluation

Georgia Southern University FRIT 7237

Richard Arenal-Mullen
&
Martha M. Hill

Implementation Evaluation
It is important to ascertain the importance of key evaluation points and evaluative
questions for the implementation of the project. Not only is it necessary to structure the input
of activities to project the output of assessment outcomes, the study must demonstrate
collaboration between faculty and mentoring of community business stakeholders and the
consultation provided from the Great Southern University faculty.
Project outcomes will provide data for accountability requirements to all stakeholders
involved, so that the design will contribute to the knowledge base of faculty and students. The
feasibility of implementing this project has determining factors upon projected impact, stage or
maturity of students, state of knowledge, impact on participating facultys classroom, and the
development of new learning materials and strategies to introduce to the classroom.
Multiple methods of data collection must be used for effectiveness and for validity. Care
to show consistency checks, differences and similarities for variation in item or scale (minimum
scoring, lowest quartile, the median, the upper quartile, and the maximum score), and pooling of
data may be used to demonstrate greater statistical power rather than with any singular posting.
The use of the demographics to show the level of socioeconomic capital should demonstrate
diversity of the testing site and the populations that are evaluated.
Notwithstanding, collection of the data should be done in a timely manner, as in this case,
less than a year, and should be mindful of the cost-efficient budget for project implementation.
Some strategies that should be stressed are reporting, briefing of individuals, interpretation of the
data collected, and the use of a pre-determined protocol for observations. This must be in the
form of evaluative rubrics that will use selective criteria for data collection while in the field.
The evaluators and participants must be mindful of project design that incorporates data
collection and the synthesis and analysis of data to communicate results after project
implementation.
The project in question requires that select teachers in the school district be given
professional learning for mathematics at the middle school level. The training consists of a five
(5) day summer workshop, on-going electronic communication, and two (2) post-workshop
meetings. For this project, the concentrated efforts will be applied to specific inputs in order to
produce the desired outcomes via several activities. The major inputs are 1) the new state
standards for math, 2) problem-based learning, and 3) the incorporation of technology. Based on
teacher concerns, standardized testing data, personnel requests, and survey data, this project was
designed to improve teacher efficacy in the area of middle school math.
For professional development impact, the project has the following activities planned
with four (4) specific outcomes. Teachers will participate in a summer conference that involves
small group work, conversations and lessons led by a field expert, and hands-on activities. Over
a period of six (6) months, teachers will communicate electronically--via wikispaces, email, and
other forms--as well as participating in two (2) post-workshop meetings. The intent of this
funding is for teachers to utilize their professional learning experiences so to enhance their
classroom experiences for student achievement on end of the year assessments.

The goals of this project are to 1) gain deeper knowledge and use of mathematics from
local business and industry, 2) create problem-based learning opportunities in the classroom
according to NSSM, 3) utilize instructional technology into PBL applications for greater gains in
assessment materials, and 4) implement PBL into instruction and evaluate the modules created
for effectiveness.
The use of a quasi-experimental method will be used to measure the participating faculty
members expertise from the project so to gather data on the one group, pre-test-posttest design.
This will enable the evaluators to measure an outcome variable based on before and after project
activities. Thus, the evaluator will hope to see the effectiveness of the program on school
assessments scores improvement, with minimal or no change to these scores after
implementation of PBL opportunities in the classroom.
Quasi-experiments for formative assessments, typically involve comparisons across two
or more conditions (before and after comparisons) and can provide compelling argument to this
procedural plan for faculty intervention and expertise when confidence may be low. By
eliminating the overkill of RCT experimental use, quasi-experiments should suffice for data of
the intended project.
Statistical power of the use of quasi-experimental method should relate a difference
between the scores attainment based on the implementation of the project and the observation of
this should be relatable to the projects impact. Qualitative approaches may be employed at this
point by interviewing project participants on the changes that they experience in the classroom
after implementation.
Summative Evaluation
Analysis of Data gathered pre-implementation:
Study focus: Braxton County to have 15,800 FTE students of varied ethnicity (e.g. 35.2%
Black, 8.9% Hispanic, 50.5% White) with 54.4% of students eligible for free or reduced lunch
achieve higher scores on standardized tests after being exposed to the PBL modules and
methods.
Teacher Interest: The mean response to this prompt was 3.9 with 61% of teachers
indicating a low-level of experience with the NSSM used in the project. This was taken from a
survey of 153 teachers in the Braxton County community. The teachers were also asked to
respond to the prompt, My interest level for participating in a professional development
experience for problem-based learning in mathematics and New Standards is using a scale from
1 to 10, with 1 indicating no interest at all and 10 indicating I want to attend. The mean
response was 6.5 with 58% of teachers indicating a high-level of interest in such a professional
development workshop.
Braxton County CRCT Data for Student Achievement (2010-2011)
6th grade data demonstrated that all students in two of the three Braxton County middle schools
failed to meet expected standards (22% - 34%). For grade 8, one in three students failed to meet
the standards.

CRCT DATA

6th grade

7th grade

8th grade

Students with
Disabilities meeting
the standard

37%

60%

33%

Black Students
meeting the standard

No scores available

69%

64%

Stakeholders involved in the Implementation Plan: Braxton County School System, Braxton
County businesses and industries, and Great Southern Universitys College of Education,
College of Science and Technology
Objectives: The goal of the project is to prepare Braxton County teachers to create and utilize
authentic, mathematics learning experiences, which incorporate the New State Standards for
Mathematics (NSSM) and available instructional technologies from the community businesses to
improve summative assessment scoring of students in Mathematics.
Activities Schedule for Grant Proposal
Day

Mon., June
25

Activity/Topic

Personnel Involved

Project
Objective

Introductions & New Standards

Teacher Participants, Great

Southern Faculty,
Consultant

Tue., June 26 Visits to local

businesses/industries for
identification of Mathematics in
context

Teacher Participants, Great

Southern Faculty, Local
businesses/industries

Wed., June
27

Problem-based learning and

Instructional Design;
Identification/review of
Mathematics in context

Teacher Participants, Great

Southern Faculty

1&2

Thur., June
28

Instructional Technology, Idea

Sharing, Problem-based Learning
module creation

Teacher Participants, Great

Southern Faculty

1, 2, & 3

Fri., June 29

Problem-based Learning Module

creation, Planning for
implementation and Evaluation
(Action Research)

Teacher Participants, Great

Southern Faculty

1, 2, 3, & 4

July 2 July

Refinement of Problem-based

Teacher Participants, Great

1, 2, 3, & 4

20

learning modules through

electronic communication

Southern Faculty

Fall 2015

Mid-implementation Meeting

Teacher Participants, Great

Southern Faculty

Fall 2015

Teachers conduct self-evaluation

of module implementation

Teacher Participants

Fall 2015 or
Early 2016

Project Debrief Meeting in

Braxton County

Teacher Participants, Great

Southern Faculty

Outcomes:
The project requires that several activities be completed to prepare teachers to utilize the
PBL modules in their classrooms. The intended impact of this project is for teachers to create a
more learner-centered environment that allows students to apply their knowledge and skills to
improve summative assessments. Furthermore, the project demonstrates the collaborative effort
between faculty and local business and industry with the emphasis on PBL and lifetime
applications. The hope is that students will improve their long-term retention of skills while
being able to reason abstractly and persevere with PBL opportunities, and the classroom
environment will demonstrate appropriate modeling of problem-based learning which will
reinforce student achievement, teacher satisfaction of assessment scoring, and stakeholder
intervention for lifetime learning opportunities.
Depending on the presentation of materials and the involvement of the individual
teachers, the quality of the project could vary. However, the planned activities seem to be of
high-quality, requiring the teachers to collaborate, synthesize, and apply knowledge in order to
create a product based on what they have learned. With those activities planned, the outcomes
should prove to be lasting and of a high quality.
At the conclusion of the project, the developed wiki with all PBL modules that were
created, will be shared with the public. All activities that were used during the development of
the project were research based and identified as a best practice for teacher professional
development and utilized specific models of instructional design and established pedagogies.
DATA COLLECTION SCHEDULE
Evaluation Plan Form
Project Director: Carlos Hedges, Ph.D.
Title of Project Proposal: New State Standards for Mathematics and Problem-based Learning

Data Set

Date of Collection

Instruments Already Data Collected by:

Developed?
Rubric for PBL
Module

June 29

Have teachers submit, Teachers and

for examination, 1)
evaluators.
the use of NSSM, 2)
inclusion of local
business and industry,
and 3) research based
pedagogical methods.

Teacher Survey of
July 1st
Workshop Experience

Have participating
teachers provide
feedback on a 5 point
Likert-Scale
regarding the group
activities, creation of
the PBL, use of
NSSM, and any
take-aways.

Evaluators

Electronic
Correspondence

Early August

Examine the number

of total
correspondences, the
perceived quality of
feedback, and any
take away the
teachers may have
collected.

Evaluators and
Teachers

Self-Evaluation

Fall 2015

Teachers will
complete a self
evaluation that
examines: 1) the
instructional gains on
in-class measurables
(tests, etc), 2) the
helpful quality and
ease of the wiki and
correspondences, 3)
the number of posts
and views on the
wiki, and 4) the
interactions, group
learning, and
connections made
during the PL.

Teachers