Sie sind auf Seite 1von 18

Semmens, R., Austin, C.M., & Riccio, G. (2010). Organizational Climate and Creation of Durable Change.

In: Riccio, G., Diedrich, F., &


Cortes, M. (Eds.). An Initiative in Outcomes-Based Training and Education: Implications for an Integrated Approach to Values-Based
Requirements (Chapter 14). Fort Meade, MD: U.S. Army Asymmetric Warfare Group. [Cover art by Wordle.net represents word frequency in
text.]

233

Semmens et al.
Chapter 14. Organizational Climate and Creation of Durable Change
Rob Semmens, C. Megan Austin
Imprimis, Inc.
Gary E. Riccio
The Wexford Group International
14.1 The Need

The Asymmetric Warfare Group (AWG) sought an independent scientific study of their initiative
in Outcomes Based Training and Education (OBTE). Two of the objectives of that effort (see
Chapter 1) are addressed briefly in this chapter:

Identify organizational factors affecting OBTE implementation (including life-cycle


management) in Army institutions of training and education.

Identify the extent to which initiatives analogous to OBTE can be developed to achieve
similar impact on other programs of training and education.

The Army has mandated change in Field Manual 1, The Army (Headquarters Department of the
Army [HQDA], 2005). It states that the Army culture needs to change to support the
transformation of the force to counter both conventional and irregular threats. While there have
been considerable gains in the implementation of OBTE over the last two years (see Chapters 1
and 11), organizational factors are likely to affect its durability (see Appendix C). Further, the
individuals and activities required to sustain OBTE are not the same as those instrumental in
initial implementation.
Early in the investigation it became apparent that there might be resistance to changing the
current approach to Army training. We thus looked for opportunities to acquire evidence for local
situational factors influencing the implementation of OBTE that were not directly under the
control of the AWG. From the outset, we were interested in answering, Why have things always
been this way? The individual officers and non-commissioned officers with whom we interacted
on the ranges and in training areas generally wanted to do the right thing. There seemed to be
some systemic constraints that did not set them up for success, however, and the apparent result
was training that had considerable room for improvement (see Chapter 11, sections 9.2-9.3).
In interviews with the AWG train-the-trainer cadre, it appeared that many common practices or
procedures had to be disregarded to demonstrate OBTE, and that organizational barriers were
prohibiting or curtailing the implementation of this approach to training (anonymous, personal
communication, April 16, 2008). Trainers in the Army have often had difficulties with Range
Control, and the demonstration of OBTE proved to be no exception. In one case, a Brigade
Commander told Range Control to go away, because he was not going to stop training to
accommodate Range Controls preferences. On another occasion, an instructor, when questioned
by Range Control about his risk assessment, had to call the post commander on his cell phone and
put Range Control personnel on the phone with the General who proceeded to approve the
training (anonymous, personal communication, May 18, 2009).

Asymmetric Warfare Group

Organizational Change

234

Sustainment of OBTE may require involvement of a wider range of personnel to reduce current
organizational frictions associated with training. Involvement of doctrine writers in the U.S.
Army Training and Doctrine Command (TRADOC), such as the working group for Regulation
350-70, and their coordination with instructor educators for OBTE is an example of the dialog
and diversity in a social network that can make a difference. Based on the organizational change
literature, even broader coordination may be required (e.g., Cooperrider, Whitney, & Stavros,
2003; Ludema, Whitney, Mohr, & Griffin, 2003; Schein, 1992).
Resistance to change is well described in the behavioral science, quality improvement, and
business literature. In the early days of social psychology, Kurt Lewin developed a model of the
change process in human systems (e.g., Lewin & Grabbe, 1945; see also, Schein, 2009). He
described resistance to change as forces in any system that hold the status quo in placeon one
side there are forces that push for change and on the other side there are forces pushing against
the change (Lewin, 1951; see Chapter 3; see also, Henle, Jaynes, & Sullivan, 1973). He
conceptualized reciprocal influences within a complex social network analogous to feedback in
electrical networks and dynamical systems (cf., Henle, 1971; Turvey Shaw, & Mace, 1982;
Riccio & Vicente, 2001). Interpersonal interactions and dialog are key manifestations of the
opportunities for such reciprocal influence (Schein, 1968; 1993).
One should distinguish the concept of resistance to change from any meaning associated with a
subordinate as a management problem or difficulties in getting a subordinate to do what the
superior wants. It demeans the concerns subordinates may have about the issues they face on the
front lines of an organization (Edmonson, 1999; Roberto, 2005, 2009). The primary locus of
friction may even be at the management level (Vaughan, 1997). In any case, friction cannot be
attributed to a single individual, subgroup, or service function. The issues surrounding a change
initiative are embedded within the complexity of an organization. Implementations of OBTE that
seem to be the most successful, at least in terms of minimal complaints about friction, seem to
address this complexity directly and through collaborative dialog (see Appendix C; Haskins,
2009; cf., Cooperrider et al., 2003; Ludema et al., 2003; Schein, 1993).
14.2 Initial Indications of Possible Resistance to Change
As reported in Chapter 10, we administered a post-deployment survey to Soldiers who had
participated in CATC to get an indication of whether the approach would be sustained in the
Operating Force. In the short survey, one question attempted to reveal indications of potential
resistance to OBTE:
Once your unit enters the next training cycle, what might you see as obstacles to
executing a CATC [OBTE] approach to some or all of your training?
While the number of respondents to the survey was small (N=17), all had been deployed in
leadership positions within a platoon. These individuals had the most to gain by this new training,
but they had little influence in planning or resourcing the training. Nine Soldiers mentioned
concerns with the larger organization being open to changing the way the Army trains. They
mentioned doctrine, safety regulations, and that some more senior echelons were resistant to
adopting a new approach or were stuck in old training standards. Five Soldiers mentioned a
possible lack of resources as an impediment, which could be interpreted as a lack of acceptance
by higher echelons in that platoons have little influence on the control the direction of resources.

Asymmetric Warfare Group

235

Semmens et al.

Another interesting indication of resistance to change was provided by a survey we administered


prior to familiarization with OBTE at Fort Sill and, again, six months after the beginning of
familiarization with OBTE (see Chapter 9).

Figure 1. Attitudes toward change at Fort Sill (see Chapter 9 for details).
The figure above indiciates that, among two different samples of officers and non-commissioned
officers, there was little change in the number of skills they thought OBTE could help them train
during basic training. All reported positive feelings about the training, however. In fact, the
Commander of the 434th BCT, indicated that their implementation of OBTE has been effective at
increasing the number of first time qualifiers (personal communication, June 30, 2009.) It is
likely that these Soldiers value the approach as useful, yet for a variety of reasons, they find it
difficult to implement formally or systemically during Basic Training. This could be due to real
or perceived organizational resistance to change (see also Chapter 9, section 9.2.3).
We continued to ask questions of those involved with implementing OBTE on an ad hoc and
informal basis. The lessons learned from this informal inquiry are used to identify leads that can
be pursued in the organizational change literature and subsequent research on OBTE.
14.3 Models and Considerations for Sustainable Change
14.3.1 The Change Transition Period
A literature review was conducted to determine similarities across several change models as an
effort to provide insight to further change implementation. While a full-scale, coordinated effort
of continuous change management cannot be outlined fully, identification of common obstacles
to change suggest activities that can be undertaken, in principle, to improve the durability of any
implementation of OBTE. The review showed that while there are many change models varying
in complexity and scope, several shared one common theme. They all describe cases where an
initial innovation found success and demonstrated a marked improvement over the standard
practice but, over time, this successful change eroded under the pressures of larger organizational

Asymmetric Warfare Group

Organizational Change

236

influences (cf., Rasmussen, 1997; Snook, 2000). Figure 2 illustrates the transition period where
change is the least durable and vulnerable to loss.

Figure 2. Influences and opportunities during the change transition period.


The change transition period has been defined in several models, yet all of the descriptions are
remarkably similar. In Kotters model (Kotter, 1995) this stage is called Consolidate
Improvements and Produce More Change. It is the seventh of eight of his stages and builds on
the sixth stage in which short-term, attainable, demonstrative improvements are made. In the
seventh stage, it is necessary to use the increased credibility from the initial success to change
systems, structures, and policies that are incongruent with the vision as well as to create an influx
of new personnel who can implement the vision and reinvigorate the change process. In the
eighth stage, the change leaders must articulate the connections between the new behavior and
success, and create new social norms and shared values consistent with changes (cf., Chrissis,
Konrad, & Shrum, 2003; CMMI Product Team, 2009). Based on findings reported in previous
chapters, it is likely that OBTE is currently in the sixth stage (although, see Appendix C for an
exception and a model for a path forward).
Several other models differentiate between the initial and sustained change. Berquist (1993)
differentiated between first-order change and second-order change. First-order change is doing
more of something or doing it better, and it is reversible. It is characterized by adjustments within
the existing structure that are not transformational; the old story can still be told. Second-order
change is considered fundamentally different in that, once the new way is in place, it becomes
impossible to return to the way things are done before. Juran (1999) agrees with this, indicating
that any change initiative actually has two change components: the intended change and the social
consequence. He defines this social consequence as the impact of the intended change on the

Asymmetric Warfare Group

237

Semmens et al.

cultural pattern of the human beings involvedon their pattern of beliefs, habits, traditions,
practices, and status symbols. This indicates a lasting and deeper implementation.
Its important to note that different kinds of influencers and implementers are needed at different
stages in a change process. Hishborn (2000) described two personalities that manifest in change
initiatives. In organizational settings, zealots provide the best leadership when the campaign's
strategic theme has yet to take root. Then, consensus builders provide the best leadership when
corporate policies need to be changed. The Diffusion of Innovations model distinguishes between
those who adopt a change early and those who adopt a change later (Rogers, 2005). The early
adopters are more open to change and more willing to take a risk. The late adopters and laggards
are likely to have fewer resources, are more skeptical of the change, and demand more evidence
before adopting the change. These different types of personalities suggest that different activities
are needed to convince different people to adopt a change (cf., Bowling, Beehr, Wagner, &
Libkuman, 2005).
14.3.2 Organizational Culture
A theme in the literature on organizational change is that an understanding of organizational
culture is of paramount importance in implementing change. Vicente (2006) describes culture as
an emergent property, the invisible hand that guides behavior.
Culture is manifested both in what people expect from one another and what people
expect from their dealings with the external environment of customers, competitors,
supplies and stakeholders. Culture is unspoken, implicit, taken for granted. You feel
the effects of culture when what you do feels appropriate or inappropriate. Culture is
largely invisible, especially to those that live within it. One of the reasons it is
difficult to change is that it is almost impossible to see the norms and expectations
that hold it in place. [p. 10]
Project failure can be closely linked to poor understanding of the targeted culture as well as the
misguided assumptions of project designers (Berger et al, 2007, pp. 121-122; Juran, 1999).
Culture must be understood and addressed if the change is to be durable (Schein, 2009). The use
of climate surveys often helps (Zohar, 2007). An understanding of culture helps disambiguate
questions that are inherently vague, such as What aspects of the Army culture are the biggest
barriers to change? Where assessment of the culture of an organization reveals friction, there
will be implications that some aspects of culture may have to change for a new initiative to be
viable and sustainable. Considering such consequences of culture, and the consequences for it,
thus can lead to the identification of new requirements. These derived requirements appropriately
supplement the requirements that originally motivated the innovation (cf., Chrissis, et al., 2003).
Rigorously traceable change proposals become contextualized, and the influence of context
becomes traceable and verifiable.
14.3.3 Clarity of Mission and Shared Understanding
A focused mission that is clearly stated and linked to a broader vision can both reflect and
influence the culture of a place (Juran, 1999). The need for change must be understood across the
organization, and that vision for the change must be maintained as newcomers join the ranks of
the change agents. Lack of understanding about the vision is the most often cited barrier to
change. Participants who understand the vision should be able to answer the question Whats in
it for me? If employees cannot explain the risks of not participating in the change, it is unlikely

Asymmetric Warfare Group

Organizational Change

238

they truly understand the mission (Prosci Research, 2007). Employees are more likely to
understand a mission if they are included as active participants in refining the definition of the
mission to ensure its viability in the local culture, consistency with a broader shared vision, and in
revealing potentially unintentional consequences (Cooperrider et al., 2003; Freeman et al., 2008;
Ludema et al., 2003; Schein, 1992, 1993; see also Appendix C).
Collaboration and communication can be utilized within an organization to achieve both buy in
and a deeper understanding of mission, objectives, and practices. This is not to abdicate
leadership. Good leaders can be directive about the process of decision making without being
directive about the solution or content of a decision; leaders can be assertive while leading with
restraint (Roberto, 2005, 2009). They can inspire leadership in others rather than thinking for
them and getting them to do what they are told.
14.3.4 Relevant Observations During the Current Investigation
Broad collaboration and communication is being used to achieve shared understanding of the
organizational climate for change in the Department of Military Instruction (DMI) at the United
States Military Academy (C. Haskins, personal communication, June 9, 2009). The Commander
of DMI has used this approach in prior assignments to build consensus around new initiatives
related to OBTE. He includes instructors and their command chain in the definition of outcomes
and in the continual adaptation of courses. He requires that the outcomes for his training be
written in plain English so that his subordinates dont apply preconceived notions to
understanding the change he is attempting to implement (Appendix C). Thus they are not merely
included. They are participants in thinking and solving problems, which is precisely what they
will try to get their students to do in the courses in DMI. Our informal observations of this
command climate suggest that it is energizing for the participants and that it motivates deeper and
broader engagement.
At Fort Sill, we were able to talk with one battery commander who had been involved with the
implementation of OBTE in his Basic Combat Training Battery. He had generally positive
feelings about OBTE as applied to marksmanship and medical skills. However, when asked if his
feedback went beyond the Battalion, he was unable to say. This may indicate a risk to the change,
as the proponents of the innovation do not have clear lines of communication with the new
adopters. More formal supports could be provided for shared understanding in the context of
organizational change (Cooperrider et al., 2003; Ludema et al., 2003; Schein, 1993; see also,
Appendix C).
14.3.5 Organizational Support and Incentives
Ultimately, even the most optimally primed environment for change must provide adequate
support. Support includes sustainment training, education, material resources, as well as time to
assimilate the new methods (Jorgensen, Owen, & Neus, 2009; Juran, 1999; National Academy for
Academic Leadership, 2007). There is a temptation for upper echelons to assume support is
adequate. Thus it is important to continue to probe lower echelons to determine if the necessary
operational and logistical resources are available to all levels in order to promote success (Berger
& Benhow, 2001; National Academy for Academic Leadership, 2007). This gives subordinates
the opportunity to comment on whether or not it is good and sufficient.
Incentives play a large role in accelerating change (Juran, 1999; National Academy for Academic
Leadership, 2007). If participants dont feel any incentive to participate they may not do so, either
actively or passively, or they may participate inconsistently. A critical consideration is that all the
Asymmetric Warfare Group

239

Semmens et al.

factors influencing an individuals behavior and performance within an organization should be


aligned. This can be complex in that implicit and indirect influences may not be well understood,
they may be different or inconsistent with explicit criteria, and they may combine with explicit
criteria to create emergent influences that may not be desirable (Rasmussen, 1997). With respect
to OBTE, all the right intentions and preparation for teaching students how to be adaptable and to
develop long-term intangible attributes easily can be undermined by an implicit or explicit
criterion based on throughput or a narrow conception of risk. For such reasons, in OBTE, it is
important to assess what instructors are doing in addition to what they are achieving, and it is
important to query them about why they are doing what they are doing.
14.4 Conclusions
The intent of this chapter was to demonstrate fruitful lines of inquiry into sources of resistance to
changes necessitated or implied by OBTE. The forces described above are the barriers all
organizations must anticipate and understand if they are to succeed in their quality improvement
initiatives. Given the common themes underlying resistance to change, we designed questions to
help identify patterns and key areas that are blocking progress at all levels involved in planning
and executing OBTE (see, e.g., Chapters 9 and 13). With these questions, we began to assess
attitudes and depth of understanding as well as to identify patterns of resistance and the highest
yield areas for improving the rate of adoption. We began to see the art of the possible in surveys
and interviews with individuals who were responsible for instruction.
Moving forward, it is clear that a thoroughgoing assessment of an instructional service system
should include assessments of organizational climate and resistance to change at all levels of an
organization that influences instruction and learning. Moreover, the intent for these assessments
should be similar to the assessments we developed for instructors (see Chapter 2) and students
(see Chapters 7 and 12). They should be replicable and actionable (Chrissis et al., 2003; CMMI
Product Team, 2009). They should provide insights that are credible, transferable, dependable,
and confirmable (Denzin & Lincoln, 2003; see Chapter 11). The multidisciplinary research
relevant to organizational change, exemplified by citations in this chapter, can be leveraged to
develop such methods of assessing organizational factors affecting change. Finally,
organizational assessments should be commensurate with the measures of instruction and
learning so that there can be a synthesis of the lessons learned. Ultimately, programmatic
decision-making requires an integrated understanding of the instructional service system with
respect to overarching goals and multifaceted ways of making coordinated progress toward those
goals (cf., CMMI Product Team, 2009; Rasmussen, 1997; Riccio & Vicente, 2001; Pellegrino et
al., 2001; see Chapters 1 and 11).
14.5 References
Berger R. & Benhow, D. (2001). The certified quality engineer handbook. ASQ Quality Press.
Bergquist, W, (1993). The Modern Organization: Mastering the Art of Irreversible Change. San
Francisco: Jossey-Bass.
Bowling, N., Beehr, T., Wagner, S., & Libkuman, T. (2005). Adaptation-level theory, opponent
process theory, and dispositions: An integrated approach to the stability of job satisfaction.
Journal of Applied Psychology, 90, 6, 10441053.
Chrissis, M.B., Konrad, M. & Shrum, S. (2003). CMMI: Guidelines for process integration and
product improvement. Boston, MA: Pearson Education.
CMMI Product Team (2009). CMMI for services, version 1.2. (CMU/SEI-TR-2009-001; ESCTR2009-001). Pittsburgh, PA: Carnegie Mellon University.

Asymmetric Warfare Group

Organizational Change

240

Cooperrider, D., Whitney, D. & Stavros, J. (2003). Appreciative inquiry handbook: The first in a
series of AI worksbooks for leaders of change. Bedford Heights, OH: Lakeshore
Communications.
Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative
Science Quarterly, 44, 350-383.
Freeman, J., Jason, J., Aten, T., Diedrich, F., Cooke, N., Winner, J., Rowe. L., & Riccio, G.
(2008). Shared Interpretation of Commander's Intent (SICI). Final Report to the Army
Research Institute for the Behavior and Social Sciences, contract number W74V8H-06-C0004.
Headquarters Department of the Army (2005). The Army. Field Manual No. 1-0. Washington,
DC: Headquarters Department of the Army.
Henle, M. (1971). The selected papers of Wolfgang Kohler. New York, NY: Liveright.
Henle, M., J. Jaynes, & Sullivan, J. (Eds.) (1973), Historical conceptions of psychology (pp. 257266). New York, NY: Springer.
Hirschhorn, L. & May, L. (2000). The campaign approach to change. Change, 32(3). 30-37.
Jorgensen, H., Owen, L., & Neus, A. (2009). Stop improvising change management! Strategy and
Leadership, 37(2), 38-44.
Juran, J. M. (1999). The quality improvement process. In J.M. Juran & A.B. Godfrey (Eds.).
Jurans Quality Handbook (5th edition). McGraw Hill, New York.
Kotter, J. P. (1995). Leading change why transformation efforts fail. Harvard Business Review,
March-April, 59-67.
Lewin, K. (1951). Field theory in the social sciences. New York, NY: Harper.
Lewin, K & Grabbe, P. (1945). Conduct, knowledge, and acceptance of new values. The Journal
of Social Issues, 1 (3), 53-64.
Ludema, J., Whitney, d., Mohr, B., & Griffin, T. (2003). The appreciative inquiry summit: A
practitioners guide to leading large-group change. San Francisco, CA: Barrett-Koehler.
National Academy for Academic Leadership (2007). Leadership & institutional change.
Retrieved April 1, 2009 from http://www.thenationalacademy.org/ready/change.html.
Pellegrino, J., Chudowsky, N., & Glaser, R. (Eds.) (2001). Knowing what students know: The
science and design of educational assessment. Washington, DC: National Academy Press.
Prosci Research (2007). Best practices in change management: Benchmarking report. Loveland,
CO: Prosci Research. Retrieved March 30, 2009 from http://www.prosci.com/tutorialresistance.htm.
Rasmussen, J. (1997). Risk management in a dynamic society: a modeling problem. Safety
Science, 27, 183-213.
Riccio, G. & Vicente, K. (2001). Coping with change and novelty in energy facility operations:
Recommendations for knowledge-based, extra-procedural problem solving. In: M. Gross &
T. Ayes (Eds.), Electrical Power Research Institute Report 1004666. Palo Alto, CA: EPRI.
Roberto, (2005). Managing for conflict and consensus: Why great leader dont take yes for an
answer. Saddle River, NJ: Wharton School Publishing.
Roberto, M. (2009). Know What You Don't Know: How Great Leaders Prevent Problems Before
They Happen. Saddle River, NJ: Wharton School Publishing.
Rogers, E. M. (2005). Diffusion of Innovations. Glencoe: Free Press.
Snook, S. (2000). Friendly Fire: The Accidental Shootdown of U.S. Black Hawks over Northern
Iraq. Princeton, NJ: Princeton University Press.
Salmoni, B. (2007). Recommendations to plenum from panel E: Military as culture. Pedagogy
for the Long WarTeaching Irregular Warfare. Quantico, VA: United States Naval
Academy and Marine Corps.
Schein, E. (1968). Personal change through interpersonal relationships. In W. Bennis, W., E.
Schein, F. Steele, & D. Berlew, D. (Eds.). Interpersonal Dynamics. Homewood, NJ: Dosey
Press.
Asymmetric Warfare Group

241

Semmens et al.

Schein, E. (1992) Organizational culture and leadership. 2d. Ed. San Francisco, CA.: Jossey Bass.
Schein, E. (1993) On dialogue, culture, and organizational learning. Organizational Dynamics,
Winter, 40-51.
Schein, E (2009). Kurt Lewin's change theory in the field and in the classroom: Notes toward a
model of managed learning. MIT Sloan School of Management. Retrieved April 16, 2009,
from http://www.solonline.org/res/wp/10006.html.
Vicente, K. J. (2006). The Human Factor: Revolutionizing the Way People Live with
Technology. Routledge, New York.
Zohar, D. (2000). A group-level model of safety climate: Testing the effect of group climate on
microaccidents in manufacturing jobs. Journal of Applied Psychology, 85(4), pp. 587596.

Asymmetric Warfare Group

ix

Riccio, Diedrich, & Cortes (Eds.)

TABLE OF CONTENTS
.

page
Prologue: A Programmatic View of the Inquiry into Outcomes-Based Training & Education.......1
Historicity of our Research on OBTE ..........................................................................................1
The Approach and Lessons Learned from the Research..............................................................3
Documentation of the Research ...................................................................................................4
Section I. Development of Stakeholder Requirements for OBTE..............................................6
Chapter 1. Preparation for Full Spectrum Operations ......................................................................7
1.1 Requirements of Full Spectrum Operations ...........................................................................8
1.2 Outcomes-Based Training and Education (OBTE)..............................................................10
1.2.1 Exemplar of OBTE: Combat Applications Training Course........................................11
1.2.2 OBTE as a Multifaceted Instructional System .............................................................12
1.3 An Appraisal of Instruction with Respect to OBTE ............................................................13
1.3.1 A Systems Engineering Framework for Integration and Development of OBTE ........13
1.3.2 Preparation for Validation and Verification .................................................................14
1.4 References ............................................................................................................................17
Chapter 2. Formative Measures for Instructors ..............................................................................20
2.1 Development of Formative Measures ..................................................................................20
2.1.1 The COMPASS Methodology ......................................................................................20
2.1.2 Development of Measures for OBTE ...........................................................................21
2.2 Description of Formative Measures .....................................................................................21
2.2.1 Results of the COMPASS Process................................................................................21
2.2.2 Elaboration on the Description of Measures.................................................................23
2.3 OBTE Performance Measures: Planning for Training.........................................................23
2.3.1 Define Outcomes ..........................................................................................................23
2.3.2 Create a Positive Learning Environment ......................................................................25
2.3.3 Create the Parameters of Learning................................................................................27
2.4 OBTE Performance Indicators: Training Execution............................................................28
2.4.1 Communicate the Parameters of Learning....................................................................28
2.4.2 Training Emphasizes Broad Combat or Mission Success ............................................29
2.4.3 Customize Instruction When Possible Based on Constraints/Conditions ....................31
2.4.4 Facilitates Learning of Concepts ..................................................................................32
2.4.5 Creates a positive learning environment.......................................................................34
2.4.6 Instructors Utilize Measures of Effectiveness & Self-Evaluation ................................36
2.4.7 Uses scenarios to facilitate learning..............................................................................38
2.4.8 Instructors exhibit intangible attributes in own actions ................................................40
2.4.9 Hotwashes and Mini-AAR............................................................................................42
2.5 Uses of the Measures ...........................................................................................................43
2.5.1 Formative Measures for Instructors ..............................................................................44
2.5.2 Quality Assurance and Instructor Education ................................................................44
2.5.3 Continuous Improvement of Assessments....................................................................45
2.5.4 Program Evaluation and Organizational Change..........................................................46
2.6 References ............................................................................................................................46

Asymmetric Warfare Group

Table of Contents

Chapter 3. Principles and Practices of Outcomes Based Training & Education............................50


3.1 Multifaceted Inquiry.............................................................................................................50
3.1.1 Interaction with Progenitors of OBTE..........................................................................51
3.1.2 AWG Documents on OBTE .........................................................................................52
3.1.3 Collaborative Reflection on Participant Observation in CATC ...................................52
3.1.4 Interaction with Stakeholders .......................................................................................53
3.2 Essential Characteristics of OBTE.......................................................................................53
3.2.1 The Meaning of Developmental is a Critical Difference..............................................53
3.2.2 The Definition of Outcomes is a Critical Difference....................................................56
3.2.3 The Emphasis on Values and Causally Potent Intangibles is a Critical Difference .....58
3.2.4 The Meaning of Experience is a Critical Difference ....................................................61
3.2.5 The Emphasis on Instructor-Student Interactions is a Critical Difference ...................62
3.2.6 The Emphasis on Learning to Learn is a Critical Difference .......................................63
3.2.7 The Emphasis on Collaborative Design and Development is a Critical Difference.....65
3.3 Toward a Grounded Theory for OBTE ................................................................................66
3.3.1 Need for an Integrated Interdisciplinary Framework ...................................................66
3.3.2 Formative Measures of Instructor Behavior as Evolving Best Practices of OBTE......67
3.4 Emerging Best Practices in OBTE for a Community-Centered Environment.....................68
3.4.1 Leadership and Enculturation of Soldiers.....................................................................68
3.4.2 Robust and Adaptable Plan...........................................................................................70
3.4.3 Instructors as Role Models ...........................................................................................70
3.4.4 Collaborative Identification of Outcomes and Measures .............................................71
3.5 Emerging Best Practices in OBTE for a Knowledge-Centered Environment .....................71
3.5.1 Integrated Understanding of Basic Soldier Skills in Full Spectrum Operations ..........72
3.5.2 Task Relevance of Planned Instructional Events..........................................................72
3.5.3 Reveal Operational Relevance of Training...................................................................73
3.5.4 Incorporate Stress into Instructional Events .................................................................73
3.5.5 Identify General Lessons Learned and Extrapolate to New Situations ........................74
3.6 Emerging Best Practices in OBTE for an Assessment-Centered Environment ...................74
3.6.1 Collaborative Reflection and Problem Solving ............................................................75
3.6.2 Communication.............................................................................................................75
3.6.3 Nature and Extent of Guidance.....................................................................................76
3.6.4 Establish a Pervasive Mindset of Collaborative Reflection..........................................76
3.7 Emerging Best Practices in OBTE for a Learner-Centered Environment ...........................77
3.7.1 Soldier Motivation and Development of Intangibles....................................................77
3.7.2 Plan for Development of the Individual .......................................................................78
3.7.3 Get Students to Take Ownership ..................................................................................78
3.7.4 Collaborative Reflection as a Means to Develop Self Efficacy....................................79
3.8 References ............................................................................................................................79
Chapter 4. Grounded Theory for Values-Based Training & Education .........................................86
4.1 Exploration of Holistic and Functionalistic Underpinnings for OBTE ...............................86
4.1.1 Fundamental Units of Analysis.....................................................................................87
4.1.2 Nested Time Scales and Adaptability ...........................................................................88
4.1.3 Adaptability and Ambiguity .........................................................................................90
4.1.4 Mechanistic Analogies and Predominant Experimental Paradigms .............................92
4.2 Three Pillars for the Scientific Foundation of OBTE ..........................................................93
4.2.1 Ecological Psychology..................................................................................................93
4.2.2 Self-Efficacy Theory.....................................................................................................97
4.2.3 Positive psychology ......................................................................................................98
4.3 A More Integrated Scientific Infrastructure .......................................................................101
Asymmetric Warfare Group

xi

Riccio, Diedrich, & Cortes (Eds.)


4.3.1 Self Determination Theory .........................................................................................101
4.3.2 Situated Learning Theory ...........................................................................................103
4.3.3 Existential Psychology................................................................................................105
4.4 Building on the Scientific Infrastructure for OBTE...........................................................109
4.4.1 Triadic Frameworks ....................................................................................................109
4.4.2 Further Development ..................................................................................................112
4.5 References ..........................................................................................................................112

Chapter 5. Passion and Reason in Values-Based Learning & Development ...............................118


5.1 The Nested Self ..................................................................................................................118
5.1.1 An Alternative to Individual versus Collective ..........................................................118
5.1.2 Cognition and Reality .................................................................................................119
5.2 Conscious Experience and the Dynamics of Thinking ......................................................122
5.3 Emotion, Information, and Engagement ............................................................................125
5.3.1 Ecological Perspective on Emotion ............................................................................125
5.3.2 Emotion as Engagement .............................................................................................126
5.3.3 Implications for Training and Education ....................................................................129
5.4 Emotion, Decision-Making, and Inter-Temporal Choice...................................................129
5.4.1 Toward a More Integrated Theory..............................................................................129
5.4.2 Emotion and Decision-Making ...................................................................................130
5.4.3 Emotion and Nested Time Scales ...............................................................................131
5.4.4 Neuroeconomics and Inter-Temporal Reasoning .......................................................132
5.5.5 Inter-Temporal Reasoning and Adaptive Dynamical Systems...................................133
5.5 Beyond Science ..................................................................................................................134
5.5.1 Existentialism..............................................................................................................134
5.5.2 The Soldier-Scholar as an Emergent Property of a Collective Pursuit.......................135
5.6 References ..........................................................................................................................137
Section II. Verification and Validation of OBTE as a Service System ..................................142
Chapter 6. Initial Impressions of Participation in CATC .............................................................143
6.1 Methods..............................................................................................................................143
6.1.1 Participants..................................................................................................................143
6.1.2 Procedure ....................................................................................................................143
6.1.3 Analyses......................................................................................................................144
6.2 Results ................................................................................................................................144
6.3 Implications for Service System Development: Peer Review ...........................................146
6.4 References ..........................................................................................................................147
Chapter 7. Local Development of Measures of Effectiveness .....................................................149
7.1 What do Instructors Believe Soldiers Should Learn in Initial Entry Training? .................149
7.2 Measure Development Process ..........................................................................................150
7.3 What do OBTE-Trained DS Believe is Important to Assess in BRM/ARM? ...................151
7.4 Implications........................................................................................................................156
7.5 Conclusions ........................................................................................................................158
7.6 References ..........................................................................................................................159

Asymmetric Warfare Group

Table of Contents

xii

Chapter 8. Observations of Behavior and Communication in Rifle Marksmanship Training .....160


8.1 Methods..............................................................................................................................160
8.1.1 Participants..................................................................................................................160
8.1.2 Procedure ....................................................................................................................160
8.1.3 Analyses......................................................................................................................161
8.2 Results ................................................................................................................................163
8.2.1 Behavior of DS ...........................................................................................................163
8.2.2 Behavior and Performance of Privates .......................................................................165
8.2.3 Patterns of Communication ........................................................................................168
8.2.4 Potential Influence of Instructor Behavior on Performance of Privates .....................170
8.3 Implications for Service System Development..................................................................171
8.3.1 Verification of OBTE .................................................................................................171
8.3.2 Validation of OBTE....................................................................................................172
8.4 References ..........................................................................................................................173
Chapter 9. Impact on Rifle Marksmanship Training....................................................................174
9.1 Behavioral Data Collection During Basic Rifle Marksmanship ........................................174
9.1.1 Method ........................................................................................................................174
9.1.2 Assessment..................................................................................................................175
9.1.3 Results An Overview ...............................................................................................177
9.1.4 Evidence for Influence of OBTE ................................................................................178
9.1.5 Behavior of Drill Sergeants after Exposure to OBTE ................................................180
9.1.6 Behavior of Privates....................................................................................................182
9.1.7 Patterns of Communication ........................................................................................186
9.1.8 Summary .....................................................................................................................186
9.2 Attitudes Toward an OBTE in Basic Training...................................................................187
9.2.1 Method ........................................................................................................................187
9.2.2 Results.........................................................................................................................187
9.4 References ..........................................................................................................................191
Chapter 10. Influence of CATC in an Operational Setting ..........................................................192
10.1 Methods............................................................................................................................192
10.1.1 Participants................................................................................................................192
10.1.2 Procedure ..................................................................................................................192
10.1.3 Analyses....................................................................................................................193
10.2 Results ..............................................................................................................................193
10.2.1 Downstream Impact on Marksmanship ....................................................................193
10.2.2 Downstream Impact on Training in the Units ..........................................................194
10.2.3 Downstream Impact on Self Efficacy .......................................................................195
10.3 Implications for Service System Development: Validation.............................................196
10.4 References ........................................................................................................................197
Chapter 11. Implications for Service System Development.........................................................198
11.1 Lessons Learned about Transfer of OBTE.......................................................................198
11.2 Implications for Service System Development................................................................199
11.2.1 Further Development and Analysis of Stakeholder Requirements for OBTE..........199
11.2.2 Further Development of OBTE as a Service System ...............................................199
11.2.3 Further Verification and Validation of OBTE ..........................................................201
11.3 References ........................................................................................................................203

Asymmetric Warfare Group

xiii

Riccio, Diedrich, & Cortes (Eds.)

Section III. Further Development of OBTE as a Service System ..........................................206


Chapter 12. Development of General Measures for Students ......................................................207
12.1 Intent ................................................................................................................................207
12.2 Performance Measure Development Process...................................................................207
12.2.1 Phase One: Define Performance Indicators (PI).......................................................207
12.2.2 Phase Two: Translate PI into performance measures ...............................................208
12.2.3 Phase Three: Measure refinement.............................................................................208
12.2.4 Phase Four: Retranslation of Measures ....................................................................208
12.3 Product of Measure Development....................................................................................209
12.3.1 Learner Perception of the Instructor and Course ......................................................209
12.3.2 Learner Engagement .................................................................................................211
12.3.3 Student Relationship with Teacher ...........................................................................212
12.3.4 Student Results .........................................................................................................214
12.3.5 Self-Report Measures ...............................................................................................216
12.4 Conclusion........................................................................................................................217
12.5 References ........................................................................................................................217
Chapter 13. Adapting OBTE in a Classroom Environment .........................................................219
13.1 Intent ................................................................................................................................219
13.2 Observing OBTE in the Classroom Environment............................................................219
13.2.1. Participants...............................................................................................................219
13.2.2. Procedure .................................................................................................................220
13.2.3. Measures ..................................................................................................................220
13.3 Utility of OBTE Measures in a Classroom Environment ................................................220
13.3.1 Generality of Measures .............................................................................................220
13.3.2. Implications for Improvement of Measures.............................................................221
13.3.3 Implications for improvement of course design .......................................................222
13.4 Use of 360 Reviews for Collaborative Reflection..........................................................223
13.4.1 The Role of a 360 Review in OBTE .......................................................................223
13.4.2 Narrative of a Participant Observer ..........................................................................225
13.5 Learning, cognitive load and motivation..........................................................................228
13.5.1 The NASA Task Load Index as a subjective measure of student workload.............228
13.5.2 Results.......................................................................................................................229
13.5.3 Implications ..............................................................................................................230
13.6 Conclusions ......................................................................................................................230
13.7 References ........................................................................................................................231
Chapter 14. Organizational Climate and Creation of Durable Change ........................................233
14.1 The Need ..........................................................................................................................233
14.2 Initial Indications of Possible Resistance to Change .......................................................234
14.3 Models and Considerations for Sustainable Change........................................................235
14.3.1 The Change Transition Period ..................................................................................235
14.3.2 Organizational Culture..............................................................................................237
14.3.3 Clarity of Mission and Shared Understanding..........................................................237
14.3.4 Relevant Observations During the Current Investigation.........................................238
14.3.5 Organizational Support and Incentives.....................................................................238
14.4 Conclusions ......................................................................................................................239
14.5 References ........................................................................................................................239

Asymmetric Warfare Group

Table of Contents

xiv

Chapter 15. Five ways OBTE can enable the Army Leader Development Strategy....................242
15.1 Background ......................................................................................................................242
15.2 An Emerging Consensus ..................................................................................................244
15.2.1 What Part to Balance?...............................................................................................244
15.2.2 Improving Training, by Design ................................................................................245
15.2.3 Increased Use of dL and Dependence on Self-Development ...................................246
15.2.4 Future Orientation, Unknown Requirements............................................................247
15.2.5 The Quality Instructor Challenge .............................................................................247
15.2.6 Purpose and Design are Key .....................................................................................248
15.2.7 A Natural Advantage ................................................................................................249
15.2.8 Task Specialization or Generalized Competency .....................................................249
15.3 Conclusion........................................................................................................................251
15.4 References ........................................................................................................................252
Epilogue. Integration of Leader Development, Education, Training, and Self-Development .....254
Toward Values-Based Standards for Army Doctrinal Requirements ......................................254
Nested Standards and Quality Assurance.................................................................................256
Needs and Opportunities for Staff & Faculty Development ....................................................259
A Role for Science and Measurement .................................................................................259
Toward Best Practices in Instructor Education....................................................................260
Critical Considerations for Further Scientific Investigation ....................................................263
The Necessity of Long-Term Studies ..................................................................................263
False Dichotomy of Objective-Subjective ...........................................................................264
Clarity About What Is Evaluated.........................................................................................265
Next Steps ............................................................................................................................266
References ................................................................................................................................268
Section IV. Appendices...............................................................................................................270
Appendix A. OBTE Principles & Practices: Instructor Measures................................................271
A.1 Genesis of Formative Measures for Instructors ................................................................271
A.2 Principles of Outcomes-Based Training & Education ......................................................272
A.3 Guide to Using Measures of Instructor Behavior..............................................................276
A.4 Complete Menu of Instructor Measures............................................................................279
Appendix B. OBTE Principles & Practices: Student Measures ...................................................318
B.1 Guide to Using Measures of Student Behavior .................................................................318
B.2 Complete Menu of Student Measures ...............................................................................319
Appendix C: A Commanders View of Outcomes-Based Training and Education .....................340
Summary ..................................................................................................................................340
Definition .............................................................................................................................340
Description...........................................................................................................................340
Elements of OBTE. ..................................................................................................................341
Developing the Outcomes....................................................................................................341
Developing the Training Plan ..............................................................................................341
Conducting Training ............................................................................................................342
How Training is Assessed....................................................................................................344
Conclusion................................................................................................................................344

Asymmetric Warfare Group

xv

Riccio, Diedrich, & Cortes (Eds.)

Appendix D: Warrior Ethos..........................................................................................................345


Analysis of the Concept and Initial Development of Applications..........................................345
Current Understanding of Warrior Ethos.............................................................................345
Purpose.................................................................................................................................348
Approach..............................................................................................................................348
Expansion of the Definition of Warrior Ethos.....................................................................348
The Tenets of Warrior Ethos ...............................................................................................349
Clarifying the Definition of Warrior Ethos..........................................................................351
Warrior Attributes Derived from the Tenets of Warrior Ethos ...........................................353
References ................................................................................................................................355
Supplementary Work Product from Warrior Ethos Project .....................................................355
Appendix E: Indicators of Warrior Ethos.....................................................................................356
Methods....................................................................................................................................356
Participants...........................................................................................................................356
Instruments and Facilities ....................................................................................................356
Procedure .............................................................................................................................356
Results ......................................................................................................................................358
Qualitative Findings.............................................................................................................358
Quantitative Findings...........................................................................................................358
Discussion ................................................................................................................................359

Asymmetric Warfare Group

Evolution of the investigation as reflected in the chapters of this monograph.