Effects of PMBOK Training on

Professional Individual
Performance Related to
Engineering and Construction
Projects Development
Renzo Toledo, MBA, PMP
Cosapi S.A.

Acerca del Autor

Renzo Toledo, MBA,

PMP

Renzo Toledo, es Gerente de

Proyectos de la Unidad de
Negocios de Ingeniera y
Proyectos EPC en Cosapi. Cuenta
con ms de 15 aos de
experiencia en proyectos de
ingeniera y construccin en los
sectores minero, energa,
hidrocarburos e industrial.
En adicin a ello es trainer y
conferencista en Gerencia de
Proyectos.

Antes que nada Advertencia

La ponencia refiere una tesis doctoral en curso
Al momento las mediciones no han sido
realizadas
El contenido de la exposicin se enfoca en el
anlisis del problema, los antecedentes en la
literatura y la propuesta del estudio a realizar
Texto en ingls Exposicin en castellano

Agenda

Introduction
Peru update
Brief story on the Research Proposal (RP)
Literature review
Doctoral Thesis Proposal (Draft)
Next steps
Conclusions (so far)

Introduction

Introduction
This paper presents an approach on the project
management training effect on actual
engineering and construction project
performance.

Introduction
First part: Literature review on several studies
conducted on this matter around the world,
and a detailed review of three of them.

Introduction
Second part: Displays a doctoral thesis
research proposal (in progress) regarding the
above mentioned topic, specifically considering
Peruvian firms and practitioners, and the
training products related to A Guide to the
Project Management Body of Knowledge
(PMBOK Guide).

Peru Update

Peru Update
Among South American countries, Peru has a
favorable combination of political stability,
represented by an civilian electoral system for
more than 30 years and a continuous economic
growth for more than 10 years.
An important investment opportunities
portfolio: mining, oil and gas, energy, metals
and machining, construction, agriculture,
fishing, airports and ports, financial services,
and tourism.
PMI is a registered trade and service mark of the Project Management Institute, Inc.
2012 Permission is granted to PMI for PMI Marketplace use only.

Peru Update
In 2011, foreign investment exceeded 15 billion
dollars (Banco Central de Reserva del Per
[BCRP], 2012).
Peru Video 1

Construction Industry
During the past few years, the construction
industry has been one of the most important
for Peruvian economic development (Instituto
Nacional de Estadstica e Informtica, [INEI],
2012), showing a continuous growth.
The construction industry fund sources are
related to new real estate initiatives, national
civil infrastructure, and private equity (local
and foreign) projects.

Peruvian Construction Industry

Peruvian Construction GDP Yearly

Variation

Peru: Mining Operations Map

13.6% territory with mining
concessions
1.09% territory in mining
operations

Peru World Mining Ranking

Zinc
#3
Tin
#3
Lead
#4
Gold
#6
Copper
#2
Silver
#2

Mining Projects Portfolio

52 Main Projects
US$ 53.4 Billion

Brief Story on the Research

Proposal

Background (Personal Quest)

February 1997 Starting in Project Community:
Engineering and Construction Industry
September 2000 MBA degree
February 2002 Metals and Machinery Industry
April 2007 PMP Certification
June 2007 Starting as PM Trainer
October 2008 Project Management VP
February 2011 Doctoral Program Application:
Thesis Theme Proposal
January 2012 Back to Engineering and
Construction

Starting Insights
Project Management Practitioner:
Are the PMBOK Guide practices actually applied?
Are they really helping me?

Project Management VP:

Are the training programs worth enough?
Do the training programs impact on performance?

Project Management Trainer:

Is the PMBOK Guide a valid framework?
How do the training programs improve practitioners
performance?

Preliminar Questions
Do the PMBOK Guide training programs actually
impact on the projects practitioners performance?
What are the related variables?
How to measure performance?
How to put this all together?

Literature Review

Background
Several project management best practices are
compiled in the PMBOK Guide. They are
recognized as the best practices used in project
development, shared by different project
management teams around the globe.

Background
The PMBOK Guide is not a methodology guide,
but a reference for project management planning
and control. The specific methodology for each
project is developed by the project team assigned
and the organization in charge (PMI, 2010, pp. 4,
1314, 3765)

Background
On the other hand, PMI (2007) proposed in the
Project Manager Competency Development (PMCD)
the professional competences required by the
project management practitioners.
The competences described are classified as:
(a) knowledge
(b)professional, and
(c) personal

Background
Knowledge competence is what the project
manager knows about the application of processes,
tools and techniques for project activities (p. 2).
Performance competence is how the project
manager applies project management knowledge
to meet the project requirements (p. 2).
Personal competence is how the project manager
behaves when performing activities within the
project environment; their attitudes, and core
personality characteristics (p. 2).

Background

Knowledge

Personal

Performance

Kirkpatrick and Phillips approach

Kirkpatrick model comprised four
dimensions:
1. Satisfaction/ReactionThoughts,
insights and feelings from the
participants regarding the
training.
2. LearningChanges in knowledge
or skills. Changes in attitudes.
This evaluation is handled during
the training (demonstrations,
presentations, tests, etc.)

Kirkpatrick and Phillips approach

3. BehaviorKnowledge, skills and
attitudes (acquired in the classroom)
applied on the job. Changes in
behavior. This evaluation is applied 3
6 months post-training, while the
trainee is performing the job.
4. ResultsResults or performance
changes (improvement) because of
attendance and participation in a
training program (can be monetary,
performance-based, etc.)

Kirkpatrick and Phillips approach

Adding to the Kirkpatrick model, Phillips (1996)
included:
5. A return on investment (ROI) evaluation level,
in order to compare the results measured in the
fourth level of the Kirkpatrick model versus the
overall costs of training.

Approach to Training Evaluation

Kirkpatrick and Phillips Model

Paper 1: Impact of interpersonal

skills training
Hunt, J. W., & Baruch, Y., (2003). Developing top
managers: The impact of interpersonal skills
training, Journal of Management Development,
22(8), 729752.
This work represents an approach to the study of
general managerial training effectiveness and
personal skills evaluation.
Program: A concentrated five-day training and
development workshop for top-level managers
(very different backgrounds, countries, and
industries).

Paper 1: Impact of interpersonal skills

training
The research focused on three questions about
training and development:
Style theorists: Are leadership skills more
relevant than technical skills?
Training and development practitioners: What is
the actual value of training and how to measure
it?
Competency theorists: Are there skills or
competencies that differentiate effective
from less effective managers?

Paper 1: Impact of interpersonal skills

training
Hypotheses
H1. The impact of interpersonal skills training
on subsequent skill performance will improve
the effectiveness of those skills.
H2. The impact of interpersonal skills training
on subsequent skill performance will be positive
but modest.
H3. The impact of interpersonal skills training
on subsequent skill performance will vary across
different skills.

Paper 1: Impact of interpersonal skills

training
180 evaluation approach: Only direct reports
(1,808) were asked to assess the training
participants.
Pre- and post-training program assesment
Paper questionnaire to record their level of
satisfaction with their managers interpersonal
skills.
29 questions, representing the skill sets already
discussed above, were ranked.
Four to six questions assessed each skill set,
using a 5-point scale: From (a) very satisfied to
(e) very dissatisfied

Paper 1: Impact of interpersonal skills

training
Results:
The statistical tests did not deny the hypotheses
The results supported the three proposed
hypotheses. The questionnaire respondents
(direct reports of the program participants),
noted various skills improved after the training
program.

Paper 2: Impact of a design management

training initiative
Bibby, L., Austin, S., & Bouchlaghem, D. (2006).
The impact of a design management training
initiative on project performance. Engineering,
Construction and Architectural Management 13(1),
726.
This work is quite closer to the typical project
management training program approach. The
authors studied the impact of a training initiative
in the Design Management Handbook (DMH) on
individual and project performance.

Paper 2: Impact of a design management

training initiative
The program consisted on a design management
training initiative to improve performance in a
major UK civil and building design and construction
company.
The research intended to measure the impact of:
(a) the DMH training program
(b)critical practices application, and
(c) a suite of 25 tools on design management
performance across the company.

Paper 2: Impact of a design management

training initiative
No hypotheses were presented:The research refers
a program assessment.
The methodology for research included:
(a) a structured questionnaire
(b)a design management maturity assessment,
and
(c) semi-structured interviews, and
(d)a case study.

Paper 2: Impact of a design management

training initiative
Results:
The study draws several conclusions regarding
the usefulness of the handbook and the
handbook application significant impact on work
performance.

Doctoral Thesis Proposal (draft)

Problem Statement
An education industry related to project
management has been rising in recent years.
Construction industrys overall investment in
project management training programs is
estimated at 5 million dollars annually for external
training programs, and a similar amount for
assigned internal resources.
There is not a study confirming that performance
has improved due to this training.

Purpose Statement
The purpose of this quantitative research study is
to determine the impact of PMI training on
Peruvian teams professional and personal
performance on engineering and construction
projects.
This study also intends to measure the impact of:
(a) The firms attitudes about training;
(b)The hierarchical level of training program
attendees; and
(c) The training program mode: internally or
externally to the company.

Conceptual Framework

Dependent Variables / Definitions

Based on Changchit, Holsapple, and Maddens (1999)
experiment design, we will define three variables:
Effectiveness: accuracy of decision making based in PMBOK
Guide training.

Efficiency: the time used to accurate decision making based in

PMBOK Guide training.
Perception: own perceptions related to the PMBOK Guide
training.

Dependent Variables

Effectiveness

Efficiency

Perception

Independent Variables / Definitions

Firm Attitude: Moderator Firm
commitment in training programs,
associated to man-hours invested for
this purpose.
Seniority Level: Moderator
Experience years in project
management.
Type of course: Mediator
Qualification of Internal or External
conducted training program.

Performance Evaluation
The individual performance to be evaluated will
be, for example:
(a) technical knowledge in project management,
(b)ability to create a Work Breakdown Structure
(WBS),
(c) capacity to define a project time schedule,
(d)accuracy for cost estimating,
(e) risk evaluation and assessment knowledge,
(f) communication techniques,
(g) interpersonal skills, among others.

Performance Evaluation
Project performance, on the other hand would
consider:
(a) project plan knowledge,
(b)project baseline deviations management,
(c) schedule accomplishment,
(d)budget control,
(e) risk responses, and others.

Research Questions / Hypotheses

Question 1: Is the current educational offer
accurate and satisfactory for the attendees? This
means that even the important training programs
offer, the actual customer expectative is
accomplished.
H1: The PMBOK Guide training program suitable
(satisfactory) for the attendees.

Research Questions / Hypotheses

Question 2: Do the attendees obtain new and
practical knowledge for their project management
activities? In addition to the question 1, this
question intends to assure if the attendees
acquired the formal concepts and knowledge.
H2: The attendees gain knowledge on Project
Management concepts and practices after the
training program.

Research Questions / Hypotheses

Question 3: Is the training attitude towards the
training program an success factor for the
individual performance improvement?
H3: The competences are improved if the firm
attitude (quantified as investment or manhours)
towards training is positive.

Research Questions / Hypotheses

Question 4: Is the training more
effective for improving new entry
level workers that senior ones? The
training should be more significant
for less experienced attendees due
to the senior practitioners should
have acquired the knowledge from
the job routine.
H4: The seniority level is inversely
correspondent to the competences
improvement after the training.

Research Questions / Hypotheses

Question 5: Is the external teaching
training course more effective that
the in-house program? Finally, is the
external training company condition is
relevant for attendees? Are the
external lecturers more effective than
the internal ones?
H5: The external teaching approach is
more effective than the internal (in
house) training.

Next Steps

Next Steps General Overview

To
To
To
To

define tools and techniques

define questionnaires
define and contact study subjects
deploy the study

Subjects of Study
Target: 300 project practitioners
Top five Engineering and Construction firms
Training programs of 80 lesson-hours at least

How to measure (E1 & E2)

In order to measure E1 and E2, two test groups will
be defined, composed of at least 50 similar project
management practitioners each. The first group
will be named CG, control group; and the second,
PMT, project management trainees.

How to measure (E1 & E2)

Testing will be performed using three typical cases
development (individual).
The first one will be applied for both groups, previous to
the PMBOK Guide training.
The second and the third typical studies will be applied
after that. The second typical case will be used as essay
and experience equalizer prior to the third typical case
solving.

A typical case is a written simulation of several

project management situations. Each question will
have four possible responses, with only one correct
response.

How to measure (E1 & E2)

The results for the groups will be compared among
groups in order to measure the PMBOK Guide
training impact among two different practitioners
groups; and among the same group, before and
after the training.
Effectiveness and efficiency will be quantified in
accordance to typical case responses to the
proposed questions.

How to measure (E1 & E2)

GROUP
CG

GROUP
PMT

TYPICAL
CASE
A

COMPARISON

COMPARISON

How to measure (P1)

For the perception variable, a questionnaire will
be given to project management practitioners with
and without PMBOK Guide training.
The results in both comparisons will be evaluated
with T-student test, as well for independent
samples as for paired samples.
The study will evaluate the influence of the
variables regarding firm attitude (FA), seniority
level (SL), and type of course (TC) with a MANOVA
test.

How to process the results

The results in both comparisons will be evaluated
with T-student test, as well for independent
samples as for paired samples.
The study will evaluate the influence of the
variables regarding firm attitude (FA), seniority
level (SL), and type of course (TC) with a MANOVA
test.

Conclusions (so far)

Conclusions
Doctoral studies on PMBOK Guide impact are
limited
Peruvian training impact studies are limited
To asses training impact is complex
To measure training impact is challenging
Tools and techniques could be hard to design
and deploy
But it is worth enough (vale la pena)
Peru Video 2

Preguntas y respuestas

Informacin de contacto
Nombre: Renzo Toledo, MBA, PMP
Compaa/Organizacin: Cosapi
Telfono: 211-3500
Email: renzo.toledo@gmail.com
Sesin: