Commission on Teacher Credentialing

Biennial Report 2014

Academic Years 2012-2013 and 2013-14
Institution Sacramento County Office Of Education
Date report is submitted September 15, 2014
Program documented in this report Multiple and Single Subject Clear Induction Program
Education Specialist Clear Induction Program
Name of Program Sacramento BTSA Induction Consortium
Please identify all delivery options
This program is delivered through traditional and
through which this program is offered
early completion options.
(Traditional, Intern, Other)
Credential awarded Clear General Education
Clear Education Specialist
Is this program offered at more than one site? No
If yes, list all sites at which
the program is offered
Program Contact

Mary Ellen Dill

Title

Director

Phone #

(916) 228-2236

E-Mail

mdill@scoe.net

If the preparer of this report is different than the Program Contact, please note contact
information for that person below:
Name

Dr. Melissa J. Neuburger

Title

Program Manager

Phone #

916-228-2575

E-mail

mneuburger@scoe.net

SECTION A CREDENTIAL PROGRAM SPECIFIC INFORMATION

PART I Contextual Information
Program Information
The Sacramento County Office of Education (SCOE) BTSA Consortium is a regional program that
partners with school districts, charter organizations, and private schools to clear teaching
credentials. SCOE administers a CTC approved Induction Program for multiple and single
subject and Education Specialist Clear Credential candidates. The Sacramento Induction
Program is a two year program aimed at providing support and assistance to teachers early in
their career. Participating Teachers (PTs) in the program can expect full support and thorough
and frequent monitoring of their progress through classroom observations, written and verbal
feedback, and assistance in implementation of the formative assessment tools used in the state
developed Formative Assessment for California Teachers (FACT), coaching, mentoring,
curriculum support, professional materials, model lessons, and communication with site
administrators regarding school site teacher needs.
Changes since Last Accrediting Activity
The structure of the program has remained the same since the last accrediting activity however
a greater focus on sharing data throughout the year occurred in 2013-14. Based on the findings
of the prior Biennial Report, the program initiated the following changes:
Program staff modeled and provided guided practice in using evidence and criteria from
the Level 2 Competency Standards to assist participant teachers in judging their practice
and identifying areas of growth.
Provided group feedback in meetings to participant teachers Individual Induction Plans
to assist support providers with developing the plans and modeling constructive
feedback for the process.
Responded to the professional development needs of support providers and participant
teachers in supporting instruction for English learners and special populations.
Streamlined processes and provided centrally located, electronic resources and tools to
assist support providers in efficiently and effectively supporting participant teachers.

Program Specific Candidate Information

Numbers of candidates and completers/graduates for two years reported
Site (If multiple sites)
Delivery Option
Traditional
Early Completion Option

2012-2013
Number of
Number of
Completers/
Candidates
Graduates
446
233
10

2013-14
Number of
Number of
Candidates
Completers/
Graduates
458
220
12

SECTION A CREDENTIAL PROGRAM SPECIFIC INFORMATION

PART II Candidate Assessment/Performance and Program
Effectiveness Information
Candidate Assessments used to Recommend Candidates for Credentials
a) Please identify and describe the tool(s) used to assess candidates, the data collection
process and the types of data collected
The Sacramento BTSA Induction Consortium utilizes several primary candidate assessments to
recommend a candidate for a credential. The assessments and measures focus on the
accomplishments and self-assessments of the candidates as they navigate through the program
and are described in the table below. Please note that in 2013-14, participant data migrated to
a new database and some participant details may not be accessible for the 2012-13 year.
Candidate Assessments
Key
Assessment
Tools

Assessment 1.

Formative
Assessment
of California
Teachers
(FACT)

Assessment 2.

Continuum of
Teaching
Practice

SCOE September 2014

Description

Data Collection Process

Standards
Assessed

This state developed formative

assessment tool is comprised of four
modules and measures participant
teachers assessment of, context for,
inquiry into, and summary of teaching
and learning through self-reflection
and review of classroom evidence in
concert with the induction experience
and with the guidance of their
support provider. The purpose is to
deepen teachers understanding of
the Inductions Program Standards
and the CTSPs. Data includes
observations, student work, lesson
plans, and reflections.

Data elements are collected

throughout each year of the
program and are reviewed by
the support provider for
sufficiency. Portfolios were
reviewed mid-year and
assessed in May of 2013 and
2014. Data for the May 2014
administration is provided.

CS 9 and PS
3, 4

This survey was administered

to participant teachers in the
fall and again in the spring for
This survey of participant teachers
the 2012-13 and 2013-14 years.
measured changes in their perceived
Participant teachers completed
competencies on representative
CS 9
the survey under the guidance
CTSPs as a results of the BTSA
PS 3, 4
program. The rating scales were taken of their support providers,
using evidence to substantiate
from the Continuum of Teaching
their level of expertise in each
Practice.
area. Answers were submitted
electronically.

Candidate Assessments (continued)

Key
Assessment
Tools

Assessment 3.

Statewide
Survey of
Participant
Teachers and
Support
Providers

Description

Data Collection Process

Standards
Assessed

This measured how well teachers

worked with their support providers
including the quality of their
relationship, frequency of meetings,
and frequency of classroom
observations. It also assessed whether
challenging conditions existed and how
those were remedied by the program or
the district and also measured the
induction and formative assessment
processes. Lastly it assessed BTSA
program preparation and impact.

This survey was

administered statewide in
spring 2013 and 2014 via a
confidential, electronic
survey. 1

CS 1, 2, 3, 6,
9
PS 1-6

Data was collected through a

portfolio and submitted
electronically in spring of
2013 and 2014. Data for the
May 2014 administration is
provided.

CS 7, 9
PS 1, 4, 5, 6

Assessment 4.

This assessment collected data for the

Education
Education Specialist credential; and
Specialist
data on the Level 2 Competency
Portfolio and Standards
Competencies

Additional Data and Measures used to Analyze Program Effectiveness and Inform
Programmatic Decision Making
b) What additional information about program effectiveness is collected and analyzed that
informs programmatic decision making?
In addition to measuring candidate competencies, the BTSA program also assesses additional
information to evaluate the programs effectiveness. The table below describes the variety of
program assessments used during the 2012-13 and 2013-14 academic years that informed
programmatic decisions. Data was collected from participant teachers, support providers, and
program coordinators and results were shared out throughout the year and posted to the
BTSASacramento.org website. Although a number of disaggregated analyses were conducted
for each assessment, only the most salient analyses are provided in this report.

After repeated efforts, the program director was unable to obtain the state survey data for the 2013-14 year
hence only the 2012-13 data is provided in this report.
1

SCOE September 2014

Program Effectiveness
Measures of
Program
Effectiveness
Assessment 1.

Participant
Teacher
Assessment
of Support
Provider
Effectiveness
Assessment 2.

Midyear
Survey of
Support
Providers and
Participant
Teachers
Assessment 3.

BTSA
Leadership
Survey

Assessment 4.

Statewide
Survey of
Participant
Teachers and
Support
Providers

Description

Data Collection Process

Standards
Assessed

This assessment collected information

about the type and level of support
received by the support provider during
that academic year. It measures meeting
frequency and the support providers
knowledge and skills in the curriculum
framework, using strategies, instructing
special populations, providing resources,
reflective conversations, guiding ILPs,
CSTPs, analysis and feedback, and using
the documents in the FACT.

This survey was

administered in spring
2013 and spring 2014 via a
confidential, electronic
survey sent to participant
teachers.

CS 6, 9
PS 3, 4

This assessment collected information

about the BTSA experience including the
availability of resources, opportunities,
and professional development provided
by the district. It also provides a midyear
snapshot of support received and
provided.

This survey was

administered in winter
2013 and winter 2014 via a
confidential, electronic
survey send to participant
teachers and support
providers.

CS 3, 6, 9
PS 1-6

This survey was

administered in spring
2014 via a confidential,
electronic survey sent to
program coordinators and
advisors at participating
school districts.

CS 1, 3, 6,
PS 1, 2

This survey was

administered statewide in
spring 2013 and 2014 via a
confidential, electronic
survey. 2

CS 1, 2, 3, 6,
9
PS 1-6

This survey collected information from

BTSA district advisors and coordinators to
rate the quality of the SCOE BTSA program
with regard to monitoring processes,
sufficiency of resources, collaboration
with districts, evaluation of professional
development, candidate preparation, and
program objectives.
This measured how well teachers worked
with their support providers including the
quality of their relationship, frequency of
meetings, and frequency of classroom
observations. It also assessed whether
challenging conditions existed and how
those were remedied by the program or
the district and also measured the
induction and formative assessment
processes. Lastly it assessed BTSA
program preparation and impact.

After repeated efforts, the program was unable to obtain the state survey data for the 2013-14 year hence only
the 2012-13 data will be reported.

SCOE September 2014

Candidate Assessment and Program Effectiveness Data

c) Include aggregated data from 4-6 assessments that were described in (a) and (b).
Candidate Assessment 1. Formative Assessment of California Teachers (FACT)
The FACT is composed of a number of requirements, all of which must be completed by the end
of the second year to complete the BTSA program. In the 2013-14 year, 241 Year 2 participant
teachers endeavored to complete the program. The table below shows that overall, 95% of
Year 2 candidates completed the program and that the key areas of insufficiency for those that
did not complete the program were inquiries, weekly conversations, and end of year
professional induction presentation.
Credential Path and Area of Incompletion for
Year 2 Participant Teachers in 2013-14 (percent
of candidates)
Completed Program
Did Not Complete the Program

Educ.
Specialist
ECO
(N = 1)
100%
0%

3%
2%
3%
2%
3%
2%
2%

36%
0%
9%
0%
0%
0%
0%

0%
0%
0%
0%
0%
0%
0%

General
Educ.
(N = 177)

94%
6%

Area of Insufficiency (Percent of candidates not completing each area)

Inquiries
Weekly Conversations
Professional Induction Presentation
Professional Development Hours
Self-Assessment
Weekly Conversations
Professional Development Courses

97%
3%

General
Educ.
ECO
(N = 11)
64%
6%

Educ.
Specialist
(N =52)

0%
6%
2%
6%
2%
0%
2%

Candidate Assessment 2. Continuum of Teaching Practice

The Continuum of Teaching Practice assessment measures participant teachers self-reported
levels of competency across six select CSTPs. Year 1 and Year 2 teachers are measured on
different select CSTPs.
Scoring rubric:
1 = emerging
2 = exploring
3 = applying
4 = integrating
5 = innovating
Participation/completion rate for all participant teacher groups combined:
In 2012-13, a total of 346 out of 456 teachers completed the survey in the fall and 230
teachers completed the survey in the spring
In 2013-14, a total of 372 out of 470 teachers completed the survey in the fall and 394
teachers completed the survey in the spring

SCOE September 2014

Total
95%
5%
4%
3%
3%
2%
2%
2%
2%

The data tables for this assessment for both 2012-13 and 2013-14 show that on average Year 1
regular education candidate teachers began the program at the exploring level and moved to
the applying level at the end of the year. Year 2 teachers began the second year of the program
at the applying level and approached the integrating level by the end of the second year.
Results for education specialist candidate teachers were similar to those of regular education
candidates with the exception that education specialist candidates rated their competency as
slightly higher, although differences in growth were not statistically different.

Continuum of Teaching Practice: Changes in Perceived

Competencies from Fall to Spring in Focus CSTP Elements
(score of 1 = Emerging to score of 5 = Innovating)

2012 Fall
N

Mean

Std.
Dev.

General Education
2013 Spring
2013 Fall
N

Mean

Std.
Dev.

Mean

Std.
Dev.

2014 Spring
N

Mean

Std.
Dev.

Year 1 Participant Teachers

1.4 Using a variety of instructional strategies, resources, and
technologies to meet students diverse learning needs
2.6 Employing classroom routines, procedures, norms, and supports for
positive behavior to ensure a climate in which all students can learn
3.5 Using and adapting resources, technologies, and standards-aligned
instructional materials, including adopted materials, to make subject
matter accessible to all students

145 2.44 0.92 96 3.24 0.79 176 2.63 0.84 183 3.24 0.90
125 2.72 0.89 87 3.41 0.70 176 2.84 0.83 183 3.31 0.82
124 2.47 1.01 87 3.03 0.87 176 2.63 0.95 183 3.21 0.94

4.4 Planning instruction that incorporates appropriate strategies to meet

121 2.46 0.94 86
the learning needs of all students

3.26 0.70 176 2.67 0.86 183 3.13 0.92

5.2 Collecting and analyzing assessment data from a variety of sources to

121 2.50 0.99 83
inform instruction

3.24 0.88 176 2.64 0.95 183 3.22 0.87

6.2 Establishing professional goals and engaging in continuous and

purposeful professional growth and development

122 2.44 0.92 86 3.32 0.79 176 2.66 1.00 183 3.25 0.97

Year 2 Participant Teachers

1.2 Connecting learning to students prior knowledge, backgrounds, life
experiences, and interests

157 3.21 0.89 101 3.80 0.79 147 2.90 0.80 157 3.62 0.78

2.4 Creating a rigorous learning environment with high expectations and

147 3.33 0.87 92
appropriate support for all students

3.89 0.78 147 2.86 0.83 157 3.65 0.81

3.2 Applying knowledge of student development and proficiencies to

ensure student understanding of subject matter

145 3.33 0.80 92 3.80 0.78 147 2.88 0.82 157 3.62 0.79

4.1 Using knowledge of students' academic readiness, language

proficiency, cultural background, and individual development to plan
instruction

144 3.31 0.90 91 3.86 0.76 147 2.93 0.85 157 3.69 0.80

5.4 Using assessment data to establish learning goals and to plan,

differentiate, and modify instruction

143 3.39 0.86 92 3.95 0.69 147 3.04 0.94 157 3.70 0.84

6.3 Collaborating with colleagues and the broader professional

community to support teacher and student learning

144 3.54 0.93 90 3.92 0.79 147 3.18 0.94 157 3.87 0.79

SCOE September 2014

Continuum of Teaching Practice: Changes in Perceived

Competencies from Fall to Spring in Focus CSTP Elements
(score of 1 = Emerging to score of 5 = Innovating)

2012 Fall
N

Mean

Std.
Dev.

Special Education
2013 Spring
2013 Fall
N

Mean

Std.
Dev.

Mean

Std.
Dev.

2014 Spring
N

Mean

Std.
Dev.

Year 1 Participant Teachers

1.4 Using a variety of instructional strategies, resources, and
technologies to meet students diverse learning needs

25 2.38 1.01 18 3.22 0.88 27 2.81 0.92 29

3.28 0.75

2.6 Employing classroom routines, procedures, norms, and supports for

positive behavior to ensure a climate in which all students can learn

22 2.82 0.73 17 3.65 0.93 27 2.96 0.85 29

3.62 0.86

3.5 Using and adapting resources, technologies, and standards-aligned

instructional materials, including adopted materials, to make subject
matter accessible to all students

22 2.19 0.81 17 3.29 1.10 27 2.89 0.93 29

3.38 0.73

4.4 Planning instruction that incorporates appropriate strategies to meet

22
the learning needs of all students

2.71 1.06 17 3.24 0.97 27 3.04 0.81 29

3.48 0.69

5.2 Collecting and analyzing assessment data from a variety of sources to

21
inform instruction

2.75 1.16 17 3.47 0.87 27 2.96 0.85 29

3.34 0.77

21 2.38 0.97 17 3.29 0.85 27 2.81 1.00 29

3.24 0.64

6.2 Establishing professional goals and engaging in continuous and

purposeful professional growth and development

Year 2 Participant Teachers

1.2 Connecting learning to students prior knowledge, backgrounds, life
experiences, and interests

19 3.25 0.77 15 3.93 0.73 22 2.73 0.77 25

3.76 0.66

2.4 Creating a rigorous learning environment with high expectations and

appropriate support for all students

17 3.71 0.69 14 4.00 0.55 22 2.73 0.94 25

3.84 0.80

3.2 Applying knowledge of student development and proficiencies to

ensure student understanding of subject matter

16 3.50 0.82 14 3.93 0.73 22 2.77 0.87 25

3.68 0.69

16 3.44 0.73 14 3.93 0.62 22 2.68 0.72 25

3.76 0.72

16 3.88 0.72 14 4.08 0.76 22 2.95 0.90 25

3.84 0.80

16 3.88 0.50 13 3.92 0.64 22 3.00 0.87 25

3.88 0.78

4.1 Using knowledge of students' academic readiness, language

proficiency, cultural background, and individual development to plan
instruction
5.4 Using assessment data to establish learning goals and to plan,
differentiate, and modify instruction
6.3 Collaborating with colleagues and the broader professional
community to support teacher and student learning

Candidate Assessment 3. Statewide Survey of Participant Teachers and Support Providers

Participant Teacher Results
At the time of this report, only the 2013-13 state survey data was available. In many instances,
the state survey data mirrored the data collected locally by the BTSA program and analyses
determined that the results were similar. For this reason, this report will not focus on the
entirely of the state survey data but rather those areas in which significant findings informed
candidate competency. At total of 417 participant teachers responded to the survey resulting
in a 91% participation rate. Scores are presented as overall means for each area. The table
below shows the relative impact of the program in each area in rank order from most impactful
to least impactful.
Scoring rubric:
1 = no impact
2 = some impact
3 = moderate impact
4 = strong impact
SCOE September 2014

BTSA State Survey Results for Participant Teachers 2012-13

How much impact did the overall BTSA Induction experience have on your
classroom practice from the following components:
Observing experienced teachers at my school or district(s)

Mean

SD

0.85

370

0.84

415

0.81

414

0.89

412

0.89

416

0.93

409

0.92

411

0.97

411

0.92

403

2.99

0.95

407

2.97

0.98

380

2.95

0.89

417

2.93

0.93

417

2.88

0.92

405

2.85

398

2.81

0.98

405

2.81

1.05

405

2.8

1.02

405

2.74

1.05

405

3.41
Coaching and feedback from my Support Provider based on observations of my
3.34
teaching and analysis of student work.
Support to develop my repertoire of teaching strategies from my Support provider
3.22
and/or professional development opportunities.
Support for managing my classroom and fostering a safe environment that promotes
student well-being from my Support provider and/or professional development
3.17
opportunities.
Designing and engaging in professional development as identified on my IIP/ILP.
3.1
Support for using results from assessment data to design instruction from my
3.09
Support provider and/or professional development opportunities.
Support in assessing student needs and differentiating instruction (including analysis
3.09
of student work) from my Support provider and/or professional development
opportunities.
Support in collaborating productively with colleagues and resource personnel, and
3.09
navigating the protocols, policies, and culture of my school and district from my
Support provider and/or professional development opportunities.
Support to develop my repertoire of assessment strategies from my Support
3.08
provider and/or professional development opportunities.
Support for teaching to content standards from my Support provider and/or
professional development opportunities.
Support forteaching English language learners from my Support provider and/or
professional development opportunities.
Collecting and analysis of evidence of my teaching practice and comparing my
teaching practice against criteria.
Development of my Individual Induction Plan/Individual Learning Plan with my
Support Provider.
Support for minimizing bias and using culturally responsive pedagogy from my
Support provider and/or professional development opportunities.
Support for teaching students with special needs from my Support provider and/or
professional development opportunities.
Support to develop my ability to collaborate with families of my students, including
communicting learning goals and progress.
Support in using technology as a teaching tool from my Support provider and/or
professional development opportunities.
Support in using technology as a learning tool from my Support provider and/or
professional development opportunities.
Support in prioritizing the professional workload.

SCOE September 2014

The state survey also asked participant teachers to identify among 15 areas where support was
desired. The areas with the highest percentage of respondents indicating it was an area of
desire support were selected for review. Results from the participant teacher 2012-13 survey
disaggregated by grade, credential type, and year in program shows that the most desired area
of support is in the development of a repertoire of teaching strategies for nearly all participant
types.

Year

Program

Grade

Subgroup

Area where support is most desired:

% Yes

K-5

b. Developing a repertoire of teaching strategies

56.4%

6-8

b. Developing a repertoire of teaching strategies

55.3%

9-12

b. Developing a repertoire of teaching strategies

64.5%

Multi-subject

b. Developing a repertoire of teaching strategies

59.3%

Single-Subject

b. Developing a repertoire of teaching strategies

59.7%

Ed. Specialist

m. Using Technology as a learning tool

63.8%

1st Year

b. Developing a repertoire of teaching strategies

64.4%

2nd Year

b. Developing a repertoire of teaching strategies

55.6%

Support Provider Results

In the statewide survey, support providers had an opportunity to indicate the extent to which
the participant teachers grew over the course of the program. The areas assessed were
comparable to those assessed of the participant teachers. Mean scores were calculated for
each area. The 2012-13 results of the 224 support providers that responded are provided in a
table below by rank order from greatest to least amount of growth. It is worth noting here that
the measurement scale on which participant teachers rated their growth was different than the
scale used by support providers. Also provided is the number of support providers that did not
know whether their participant teachers grew in the respective area or not.
Scoring rubric:
1 = no observable growth
2 = little observable growth
3 = some observable growth
4 = significant observable growth

SCOE September 2014

BTSA State Survey Results for Support Providers 2012-13

Based on your work with participating teachers, to what extent did they
grow in their practice in the following areas as a results of their
participation in BTSA?

Mean

SD

Number
that did
not
know

Developing a repertoire of teaching strategies

3.62

0.56

Collaborating productively with colleagues

3.60

0.61

Managing the classroom and fostering a safe environment

3.58

0.62

Teaching to content standards

3.57

0.56

Assessing student needs and differentiating instruction

3.54

0.55

Using assessment data to design instruction

3.50

0.60

Ensuring access to the curriculum for all students

3.49

0.56

Using technology as a teaching tool

3.40

0.68

Developing a repertoire of assessment strategies

3.39

0.56

Collaborating with families, including communicating assessment results

3.34

0.72

14

Using technology as a learning tool

3.32

0.74

10

Teaching students with special needs

3.29

0.72

25

Prioritizing the professional workload

3.29

0.66

11

Minimizing bias, and using culturally-responsive pedagogy

3.26

0.69

13

Teaching English language learners

3.19

0.80

17

Candidate Assessment 4. Education Specialist Portfolio and Competencies

During the 2013-14 program year, 100% of the 52 Year 2 candidates working on their Level 2
competencies had completed the level 2 competencies specific to this credential. The three
that had not completed the entire BTSA program by Year 2 were lacking completion of other
requirements as was noted prior.
Program Effectiveness Assessment 1. Participant Teacher Assessment of Support Provider
Effectiveness
This assessment asks participant teachers to rate the knowledge and support of their support
providers. Data for Year 1 and Year 2 participant teachers and data for credential type were
combined to determine the overall measure of support received.
Scoring rubric:
1 = weak
2 = average
3 = strong
Participation/completion rate for all participant teacher groups combined:
In 2012-13, a total of 388 out of 456 teachers completed the survey
In 2013-14, a total of 414 out of 470 teachers completed the survey
SCOE September 2014

10

On average, scores for both years indicate that average to strong support and knowledge was
present in the support providers. In the 2014 year, a statistically significant improvement was
made (bright green shading) in the understanding of program requirements by support
providers to effectively assist participant teachers in their completion of the program.
Participant Teachers Rating of Support Providers' Knowledge and Skills in
the Following Areas (score of 1 = weak to score of 3 = strong)
1. Using our meeting time effectively
2. Sharing behavior management strategies
3. Sharing strategies and resources for teaching English learners
4. Sharing strategies and resources for addressing the needs of special populations
including GATE students
5. Identifying instructional strategies and materials appropriate to my teaching
context
6.Creating a trusting relationship with me
7. Using reflective conversation techniques
8. Helping me develop an Individual Induction Plan (IIP) based on assessment
evidence
9. Analyzing student work
10. Reviewing the information from a classroom observation and providing
constructive feedback
11. Understanding the "Plan, Teach, Reflect, and Apply Cycle"
12. Assisting me in understanding my context for teaching
13. Using the state-adopted content standards, curriculum frameworks, and contentspecific pedagogy to improve the performance of my students
14. Assessing my teaching practice based on the California Standards for the Teaching
Profession
15. Helping me select professional development that is aligned to my IIP
16. Using the documents in the Formative Assessment for California Teachers (FACT)
17. Understanding the requirements for me to complete this program and clear my
credential

Year

Mean

Std.
Dev.

2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014

388
414
388
414
387
413
387
413
388
414
388
413
388
412
388
414
388
414
388
413
388
413
387
412
388
414
388
414
387
413
387
414
388
414

2.79
2.81
2.74
2.75
2.57
2.62
2.61
2.66
2.78
2.79
2.85
2.87
2.79
2.81
2.72
2.78
2.74
2.74
2.79
2.80
2.71
2.76
2.74
2.79
2.72
2.72
2.78
2.79
2.68
2.70
2.60
2.60
2.74
2.83

0.47
0.43
0.51
0.51
0.60
0.58
0.59
0.56
0.50
0.48
0.44
0.41
0.49
0.46
0.56
0.47
0.49
0.50
0.49
0.46
0.54
0.47
0.54
0.46
0.51
0.50
0.47
0.45
0.57
0.55
0.61
0.58
0.53
0.43

Sig.

.523

.470

.062

.804

1.498 .221
1.438 .231
.300

.584

.763

.383

.257

.613

2.604 .107
.003

.954

.025

.874

1.872 .172
1.770 .184
.000

.991

.039

.844

.200

.655

.027

.869

6.062 .014

It was also noted that participant teachers who did not meet regularly with the support
providers throughout the program rated their support providers as weak to average in nearly all
areas measured for both years reported. These results are illustrated in the figure below.

SCOE September 2014

11

Program Effectiveness Assessment 2. Midyear Survey of Support Providers and Participant

Teachers
The midyear surveys measured the frequency of program tool use, the BTSA experience, and
the support provided and received by participant teachers and support providers. Results for
each survey are presented separately.
Support Provider Survey Results
Scoring rubric:
1 = do not agree
2 = slightly agree
3 = moderately agree
4 = strongly agree
Participation/completion rate for support providers:
In 2012-13, a total of 199 out of 212 support providers completed the survey
In 2013-14, a total of 231 out of 243 support providers completed the survey
The table below depicts the percent of support providers that are using the program tools.

SCOE September 2014

12

Use of Program Tools for Self-Reflection and Development

2013
% Yes

2014
% Yes

Are you using the Continuum of Teaching Practice to assess the progress of your participating
teachers?

199

100%

231

99%

Have you and your participating teacher(s) revisited, reflected on, and updated a professional
growth plan (IIP)?

199

93%

231

86%

Are you and your participating teacher(s) able to review the results from classroom observation
and assessment evidence in a timely manner?

199

96%

231

95%

Have you led your participating teacher(s) in an assessment of their own teaching practice?

199

97%

231

96%

Have you and your participating teacher(s) developed and used instructional strategies based on
state-adopted academic content standards and students' performance levels?

199

98%

228

99%

Has the BTSA program provided you with the opportunity to reflect on your own level of
practice and plan for your own personal professional growth?

199

97%

228

99%

Do you have an action plan for your own professional growth?

199

94%

228

92%

The midyear survey also asked support providers about their BTSA experience. The table below
shows the average level of agreement to each statement about the program and whether any
statistically significant differences (*p=.05) were found between the 2012-13 and 2013-14
program years (bright green highlight).
Level of Agreement by Support Providers: ANOVA Comparison of Program Years
Experience as a BTSA Provider
a) I understand the requirements for completing the BTSA Induction Program and earning a
clear teaching credential.

Year

2013
2014
2013
b) Professional development in my school and district align well with my BTSA activities.
2014
c) My district provides adequate resources and support to enable me to fulfill my role as a
2013
support provider.
2014
d) My participating teachers have sufficient opportunities to work with special education
2013
teachers to support their students.
2014
e) My participating teachers have sufficient access to technology that supports teaching and 2013
student learning.
2014
f) I have sufficient time to meet with peers to develop and refine my support provider skills, 2013
engage in problem solving, and reflect on teaching.
2014
Support Provided to Participant Teachers
a) Additional and/or special assistance to meet the unique challenges of their teaching
2013
assignment.
2014
2013
b) Guidance based on evidence from their teaching practice.
2014
c) Professional assistance in using evidence and assessments to improve their teaching
2013
practice.
2014
Reflections on the Sacramento BTSA Consortium
a) Provides effective support via training, peer support meetings, and other professional
2013
development.
2014
2013
b) Has positively impacted my own professional growth.
2014

SCOE September 2014

13

Mean

SD

199
228
199
228
199
228
199
228
199
228
199
228

3.75
3.73
3.40
3.30
3.44
3.35
3.01
2.99
3.32
3.26
3.23
3.07

.51
.53
.76
.78
.80
.77
.91
1.02
.84
.81
.81
.88

199
228
199
228
199
228

3.60
3.49
3.74
3.62
3.67
3.57

.57
.62
.47
.54
.53
.56

199
228
199
228

3.56
3.54
3.53
3.54

.66
.67
.72
.72

Sig.*

.261

.609

1.918

.167

1.415

.235

.040

.841

.523

.470

3.583

.059

4.047

.045

5.515

.019

3.707

.055

.081

.776

.001

.972

Participant Teacher Survey Results

The participant teacher midyear survey focused on the participant teachers BTSA experience,
support they received from their support providers, and their perceived increases in
competency as a result of the program. For these analyses, data for Year 1 and Year 2
participant teachers and credential types were combined.
Scoring rubric:
1 = do not agree
2 = slightly agree
3 = moderately agree
4 = strongly agree
Participation/completion rate for all participant teacher groups combined:
In 2012-13, a total of 424 out of 456 teachers completed the survey
In 2013-14, a total of 448 out of 470 teachers completed the survey
The table below shows the average level of agreement to each statement about the program
and whether any statistically significant differences (*p=.05) were found between the 2012-13
and 2013-14 program years (bright green highlight).

SCOE September 2014

14

Participant Level of Agreement: ANOVA Comparison of

Program Years
Participant Teachers' BTSA Experience
a) I understand the requirements for completing the BTSA
Induction Program and earning a clear teaching credential.
b) My district has provided adequate resources and support
to enable me to become a better teacher.
c) I have sufficient opportunities to work with special
education teachers to support my students.
d) Professional development in my school and district align
well with my BTSA activities.
Participant Teachers' BTSA Support Providers
a) Is a skilled mentor and guide.
b) Effectively uses the Formative Assessment for California
Teachers (FACT) to help me examine my practice.
c) Effectively guides the development and implementation
of my Individual Induction Plan (IIP).
d) Effectively uses evidence and assessments to guide our
work together.
e) Shares useful instructional ideas and materials with me.
BTSA Improved Participant Teachers' Competency
a) Classroom management.
b) Assessing students specific learning needs.
c) Planning and differentiating instruction to meet the
diverse learning needs of the full range of learners.
d) Teaching and supporting English Learners.
e) Designing and implementing an equitable and inclusive
learning environment in my classroom.
f) Maximizing academic achievement for students from
various backgrounds and experiences.
g) Using a variety of resources to collaborate with students
and their families.
h) Using assessment data to monitor student progress.
i) Communicating with students families about their
progress.

Year

Mean

SD

2013
2014
2013
2014
2013
2014
2013
2014

424
448
424
448
424
448
424
448

3.59
3.64
3.46
3.51
2.94
3.04
3.26
3.41

0.59
0.59
0.73
0.66
0.99
1.01
0.82
0.77

2013
2014
2013
2014
2013
2014
2013
2014
2013
2014

424
447
424
447
424
447
424
447
424
447

3.75
3.81
3.56
3.63
3.63
3.70
3.62
3.71
3.76
3.81

0.55
0.47
0.71
0.66
0.68
0.61
0.69
0.57
0.58
0.51

2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014
2013
2014

424
446
424
446
424
446
424
446
424
446
424
446
424
446
424
446
424
446

2.90
2.93
2.94
3.00
2.96
3.06
2.64
2.78
2.98
3.05
2.89
2.99
2.83
2.90
3.02
3.06
2.72
2.78

0.90
0.92
0.85
0.86
0.83
0.85
0.92
0.91
0.84
0.87
0.86
0.87
0.88
0.93
0.85
0.84
0.93
0.93

Sig.*

1.48

.225

1.52

.217

2.13

.144

7.74

.006

3.44

.064

1.96

.162

2.26

.133

4.54

.033

1.89

.170

0.35

.555

0.88

.349

3.41

.065

4.96

.026

1.21

.273

3.30

.069

1.35

.245

0.47

.492

0.93

.336

Program Effectiveness Assessment 3. BTSA Leadership Survey

The BTSA Leadership Survey was administered in spring of 2014 to garner feedback from
district BTSA coordinators and advisors about the support that SCOE provides to them with
regard to program resources and candidate preparation. The table following the scoring rubric
shows the average program rating for each item listed.
SCOE September 2014

15

Scoring rubric:
1 = low
2 through 4 (not defined by a text descriptor)
5 = high
Participation/completion rate for district coordinators and advisors combined:
In 2013-14, a total of 16 out of 39 district coordinators and advisors completed the
survey
BTSA Leadership Survey Results: Program Ratings for 2013-14 by Coordinators
and Advisors
1. The SCOE program has a clear, researched-based vision for the induction of new
teachers which is articulated with stakeholders.
2. The SCOE program leaders and relevant stakeholders are highly involved in the
organization, governance, and coordination of this program.
3. SCOE leaders have strong support from the district.
4. SCOE has a well-defined and monitored process for ensuring that all
participating teachers have met all credential requirements.
5. SCOE provides sufficient funding, personnel, and facility resources are
consistently allocated to this program to enable effective operation and support.
6. The resource needs of the SCOE program are regularly reviewed, evaluated, and
updated.
7. SCOE regularly evaluates the performance of professional development
providers and seminar facilitators.
8. The activities of the SCOE program are a logically sequenced extension to
participating teacher pre-service learning.
9. There is close collaboration between the SCOE program and district
administration.
10. The requirements for the participating teacher to complete this Induction
program are clear.
11. The participating teachers in the SCOE program have opportunities to learn
about the application of technology to student learning.
12. Participating teachers have resources and opportunities that will assist them in
improving their skills in developing strategies for teaching English learners.
13. Participating teachers have resources and opportunities that will assist them in
improving their skills teaching students that have disabilities, are at risk, or are
gifted and talented.
14. Participating teachers have resources and opportunities in the areas of
differentiating instruction to meet the various needs of students.
15. There is sufficient evidence regarding candidate progress and performance to
guide advisement and assistance efforts.
16. The SCOE program provides candidates sufficient opportunities to develop
research-based strategies for improving student learning.
17. The SCOE program prepares candidates with the proficiencies and
competencies to educate and effectively support all students.

SCOE September 2014

16

Mean

Std.
Dev.

16

4.81

0.54

16

4.88

0.34

16

4.75

0.58

16

4.94

0.25

16

4.81

0.40

15

4.93

0.26

14

4.79

0.58

16

4.88

0.50

16

4.31

0.87

16

4.94

0.25

15

4.60

0.74

15

4.47

0.74

15

4.47

0.74

15

4.60

0.51

16

4.88

0.34

16

4.75

0.45

16

4.75

0.45

Program Effectiveness Assessment 4. Statewide Survey of Participant Teachers and Support

Providers
As noted prior, at the time this report was written, state survey data was only available for the
2012-13 year. Data from the state survey is extensive however the data provided for this
assessment will be limited to the most significant findings for the program, especially since very
similar data is collected locally and analyzed with greater detail.
Participant Teacher Results
The statewide participant teacher survey asked teachers about the induction and formative
assessment process. The table below shows the results for the 2012-13 participant teacher
survey with regard to the connection between their IIP/ILP and the BTSA program. A total of
417 participant teachers responded to the survey.
Scoring rubric:
1 = no connections
2 = some connections
3 = moderate connections
4 = strong connections
BTSA State Survey Results for Participant Teachers
2012-13
How strong were the connections between the goals
and research activities you developed for your
Individual Induction Plan/Individual Learning Plan and
the following:
Preliminary credential preparation goals and activities
(teacher preparation)
BTSA Induction professional development activities

Percent of Respondents (N = 417)

Strong
Connections

Moderate
Connections

Some
Connections

No
Connections

Mean

28.7%

44.5%

24.2%

2.4%

3.00

21.5%

41.4%

31.3%

5.3%

2.80

District/site professional development activities

25.1%

41.1%

27.0%

6.5%

2.85

Work with your BTSA Induction Support Provider

53.1%

31.6%

13.4%

1.4%

3.37

District formal teacher evaluation process

21.1%

36.1%

27.8%

14.6%

2.64

Support Provider Results

Like the participant teachers, support providers responded to a similar state survey. A total of
224 support providers responded to the statewide survey of support providers.
In comparing the demographics of state support providers to the SCOE support providers, some
important differences were noted. As compared to the state, more SCOE support providers:
Carried a full time teaching load (4% percentage points more)
Had additional duties that impacted their support time (4% percentage points more)
Supported two or more other interns/teachers in addition to the BTSA teachers (3%
percentage points more)
Even with a slightly greater burden upon them however, SCOE support providers believed that
they provided the same level of service as the other providers in the state.
SCOE September 2014

17

The results of the support provider survey that are presented below focus on identified needs
and areas of support, rather than a review or comparison of all survey questions. In order for
support providers to support participant teachers, support providers must also have the
appropriate knowledge and skills. The figure below depicts the percent of support providers
that have not received professional development in each of the areas noted. The areas are
ranked in order from low to high areas of need. The orange and blue sections highlight areas in
which nearly 20% and 30% of support providers, respectively, did not receive professional
development.

The 2012-13 statewide survey also asked support providers to identify the areas in which they
desired more professional development. The table below depicts the top areas cited for
professional development for first year support providers and includes data for second year and
more experienced support providers (3 or more years) as well.
SCOE September 2014

18

Professional Development Opportunities Desired (Most Popular

Listed for Year 1 Support Providers)

Years as a Support
Provider

Percent of support providers desiring professional development in

each area.
Developing strong, relevant and powerful Individual Induction
Plan/Individual Learning Plans (IIP/ILP)

Year 1
SP

Year 2
SP

Year
3+ SP

77.0%

78.8%

55.8%

Building personal leadership capacity

73.8%

45.5%

42.5%

Developing strong, relevant and powerful inquiry questions

72.1%

72.7%

63.3%

Observing and giving feedback

72.1%

63.6%

53.3%

Networking with other Support Providers

70.5%

69.7%

55.0%

Note: areas in purple were not a top need identified by Year 2+ support providers, data
is simple provided to lend continuity

PART III Analyses and Discussion of Candidate and Program Data

Describe what the analyses of the data demonstrate about your program relative to: a)
candidate competence; and b) program effectiveness.
Overall, results from the candidate and program assessments indicate that the BTSA program is
effectively meeting its goals and that it provides appropriate support to participant teachers to
enable them to complete the program requirements and clear their credential.
Candidate Assessment 1. Formative Assessment of California Teachers (FACT)
In reviewing the data from the FACT, it was evident that nearly all Year 2 participant teachers
complete the program within two years. For those that did not complete the program by the
end of the second year, the greatest barriers were completion of the inquiry cycles, weekly
conversations, and end of year induction presentation. Given that the inquiry cycles are the
forefront of the other requirements, candidates who do not successfully complete these on
time, quickly fall behind in the program. Those teachers utilizing the early completion option
were most challenged with meeting the inquiry cycles and end of year induction presentation
requirements, suggesting that working with a support provider is more successful than
attempting to complete the program on their own.
Candidate Assessment 2. Continuum of Teaching Practice
Results for the Continuum of Teaching Practice assessment showed that Year 1 participants
reported on average one level of growth from the fall to the spring while Year 2 participants
reported growing half a level on the CSTPs as a result of the program. Initial competency levels
were slightly lower for the 2012-13 participant teachers than for the 2013-14 participant
teachers suggesting that the pool of teachers in the second program year had slightly more
initial experience than candidates from the prior year. Results also indicate that the 2013-14
Year 1 teachers (both general education and education specialists) were strongest in the area of
employing classroom routines while Year 2 teachers indicate that their strength occurred in the
SCOE September 2014

19

realm of collaboration. Year 1 teacher weaknesses lay in the areas of differentiating instruction
to meet the needs of special populations. By the end of Year 2 however, scores in most of the
lower scoring areas tended to level out thus relative weaknesses were less substantial.
Worth noting is that although candidates tended to over-report their levels of competency,
they recognized that much professional growth occurred throughout the program. As a
measure of teacher competency, this assessment tool not only informs the teacher about their
practice and the use of evidence, but also engenders a deeper understanding of the standards
for the teaching profession that guides their growth as teachers.
Candidate Assessment 3. State Survey
The state survey provided confirmation of the results found in the local program assessments.
Teachers cited that the program elements of observing experienced teachers, receiving
coaching and feedback, and developing a repertoire of strategies to use in the classroom had
the biggest impact on their development as a teacher. The least impactful elements of the
program were in prioritizing the professional workload and in using technology for teaching and
learning. Not surprising, participant teachers cited that the areas of professional development
they desired most were developing a repertoire of teaching strategies (all teachers) and using
technology as a learning tool (education specialists).
Self-reported growth of teachers in the area of developing a repertoire of strategies was also
confirmed by the statewide survey data as this was the area support providers indicated that
teachers grew the most. Support providers reported that teacher grew least in the area of
teaching English learners and in minimizing bias and using culturally responsive pedagogy.
These results align with the data from the Continuum of Teaching Practice and the selfidentification of professional development needs by participant teachers.
Candidate Assessment 4. Education Specialist Portfolio and Competencies
Based on the few candidates whose requirements were not completed, it can be deduced that
completion of the additional portfolio and competencies was not the sole reason that
candidates did not complete the program. This area would not be considered as an area
identified for improvement over the course of the next cycle.
Program Assessment 1. Participant Teacher Assessment of Support Provider Effectiveness
Participant teachers rated their support providers knowledge and skills as being average to
strong across the 17 areas of support. Ratings of support providers understanding of the
requirements needed to complete the program and clear the credential showed significant
improvement from spring 2013 to spring 2014. Support providers greatest strength lay in
creating a trusting relationship with the participant teacher and the area needing greatest
improvement was in sharing strategies and resources for teaching English learners. Not
surprising, having regular meetings between support providers and their participant teachers
was found to be essential to program support.

SCOE September 2014

20

Program Assessment 2. Midyear Survey of Participant Teachers and Support Providers

The major finding of the midyear survey of support providers was that in nearly all cases,
support providers were consistently utilizing the program tools with their participant teachers.
In instances where this did not occur, support providers cited that unexpected absences (e.g.
maternity leave) prevented the full use of the tool or cited that a plan was in progress. Areas in
which support providers could improve were: the provision of increased opportunities for
participant teachers to work with special education teachers; assistance to meet the unique
challenges of the participant teachers teaching assignments; and providing guidance to
participant teachers based on evidence of their teaching practices.
Results from the participant teacher midyear survey showed that participant teachers were
pleased with their BTSA experience and the support they received from their support providers.
They also cited that participation in the BTSA program increased their teaching skills and
classroom environment, especially in the use of assessment data and differentiating instruction.
Skill areas identified for improvement from this assessment are teaching and supporting English
learners and also communicating with families about students progress.
Program Assessment 3. BTSA Leadership Survey
Results from the BTSA leadership survey indicated that the SCOE program and staff effectively
served the needs of program coordinators and advisors during 2013-14. Results from the spring
2014 survey show high ratings for all areas of program support, particularly in the clarity of the
requirements to complete the induction program and having a well-defined monitoring
process. Participants indicated (through open ended questions) that the program could be
improved by increasing involvement of site and district administrators in the program and by
having a summer training for new district coordinators to help them learn their new role.
Program Assessment 4. State Survey
Data from the participant teacher statewide survey indicated that the connections between the
goals and research activities teachers developed for their IIP/ILP were strongest in the areas of
working with their BTSA Induction support provider and also in the preliminary credential
preparation goals and activities. The weakest connection was found in the district formal
teacher evaluation process, suggesting again that a closer relationship between the program
and the district would be beneficial.
With regard to the statewide support provider survey, one of the most profound findings was
that although most providers had the requisite skills to support their teachers in most areas,
support providers also needed professional development in some of the same areas that
participant teachers cited. As was highlighted in the orange shaded area on the chart, nearly
20% of support providers had not received professional development in the areas of:
Working effectively with English learners
Working effectively with special needs students,
Guided group analysis of participant teacher or support provider artifacts
Group training on professional learning communities
SCOE September 2014

21

Areas in which 30% of support providers said they had not received professional development
were highlighted in the shaded blue area on the chart and include:
Using technology
Coaching from BTSA program director or coordinator
Analysis of survey data
Peer coaching with other support providers
Written feedback
The state survey also asked providers to identify areas of professional development that would
enhance their support provider knowledge and skills. The largest areas in which professional
development had been desired were all focused on providing better support to their participant
teachers, including opportunities for observation and feedback, collaboration, and supporting
the ILP/IIP and inquiry processes. If providers are to effectively support their participant
teachers in these same areas, then they need access to professional development as well.
PART IV Use of Assessment Results to Improve Candidate and Program Performance
1-2 pages
In many cases, the data from the various assessments led to the same conclusions. A summary
of the proposed programmatic changes based upon the findings of the assessment tools are
described in the tables that follow.
Data Source

Plan of Action or Proposed Changes Made

Applicable
Program or
Common
Standard(s)

FACT

Candidates seeking to enroll in the early completion

option will only be allowed to do so after thorough
discussion with the candidates regarding the difficult
nature of completing the program requirements without
the benefit of a support provider. Early candidate
completion options must also be approved by the
candidates school district. We will ensure the ECO
process which has been developed is clearly defined to all
stakeholders.
The depth and complexity of the program activities will
be differentiated to better address the greater initial
knowledge and experience of Year 1 participant teachers.
In addition, the CSTPs and the Continuum of Teaching
Practice will be directly reviewed with participant
teachers at benchmark meetings to ensure their full
understanding of the standards to better anchor their
self-reported levels of competency.

CS 9
PS 3, 4

Continuum of
Teaching Practice

SCOE September 2014

22

CS 9
PS 3, 4

Data Source

Plan of Action or Proposed Changes Made

Statewide Survey
of Participant
Teachers and
Support Providers

After sharing results of the survey to constituent groups, CS 1, 2, 3, 6, 9

PS 1-6
it became apparent that support providers desired
structured feedback about the support they provided and
desired more opportunities for collaboration. In fall of
2013 support providers and participant teachers began
developing a feedback worksheet containing content
areas for feedback, method of delivery and frequency of
administration. The final feedback form will be
completed for use during October 2014 benchmarks.
Furthermore workshops agendas were designed to
support collaboration time in 2013-14 and will continue
to do so in subsequent years.

Participant
Teacher
Assessment of
Support Provider
Effectiveness

It was evident that regularly scheduled meetings

between participant teachers and support providers was
essential to program success. To this end, the program
advisors and coordinators will be expected to follow up
with the support providers at their site to remedy any
scheduling issues that may arise. Coordinators will
report, discuss, and resolve their monitoring of regularly
scheduled meetings and record this in the BTSA
database.

CS 6, 9
PS 3, 4

Midyear Survey
of Participant
Teachers and
Support
Providers,
Statewide Survey
of Participant
Teachers and
Support
Providers, and
Assessment of
Support Provider
Effectiveness

To address the need for greater resources to support

English learners and to increase resources and
opportunities to work with special needs students, fall
2013 meeting time was dedicated to creating a
centralized list of resources (My BTSA Resources) that
was published to the MyBTSA website. A continued goal
for the 2014-15 year will be to identify sources and
provide guidance and expertise for professional
development for both support providers and participant
teachers. To better assess program support for students
in these areas, the FACT was redesigned for the 2014-15
year to incorporate elements for English learners and
special populations.

CS 1, 2, 3, 6, 9
PS 1-6

SCOE September 2014

23

Applicable
Program or
Common
Standard(s)

Data Source

Plan of Action or Proposed Changes Made

BTSA Leadership
Survey

CS 1, 3, 6,
To increase the involvement of site and district
PS 1, 2
administrators in the program, additional activities will
occur in the 2014-15 year. These activities include: a
survey to determine how information from the program
can best support their program teachers needs as well as
ways in which they would value further involvement in
the program; additional invitations to participate in BTSA
meetings; and more frequent communication with
district and site administrators about program results and
activities based on survey result findings.

SCOE September 2014

24

Applicable
Program or
Common
Standard(s)