Certificate in

Learning & Development Practice

Assessed Coursework Cover Sheet

Student Name : Rodzidah bt Mohd Rodzi

IC : 791103-10-5336
Organisation : CIAST, Shah Alam
Unit Title : Understanding Organisations & The
Role of Human Resources

Unit Code : 3ELA F212B

Coursework Component : Written Assignment
Date of Submission : 3 July 2012

1
 Activity 1

Write a report about the nature and purpose of evaluation, which explains :

Question 1

1. How evaluation differs from / relates to : validation, assessment, monitoring and

evaluation.

Answer

How evaluation differs from/ relates to : validation, assessment, monitoring and evaluation.

Measuring Training
What it means?
Effectiveness

All analytical methods used to validate the training

Validation
programs in respect to accuracy

The action of assessing the effectiveness of the

Assessment
training programs

Observe and check the progress or quality of the

Monitoring
training programs over a period of time

The making of a judgment about the value of the

Evaluation
training programs

2
 Question 2

2. The purposes of evaluation for 3 different stakeholders

Answer

Purpose of evaluation ( to identify training effectiveness by answering all the

stakeholder needs)

L = Levels of Evaluation

Who? What do you want to know?

General Manager The achievement of :

-vission, mission, objective

-Return On Investment (ROI)

-Return On Expectation

Level 4, 5 & 6

Head of Departments The Planning of :

-budget allocations

- Key Result Area (KRA) performance

-Customer satisfaction

-cost effectiveness

- Level 4 & 5

Managers The trend and behavior of :

-Attitude, Skills. Knowledge (ASK) gap

- Competence qualified trainers ,

- Key Performance Indicator (KPI) achievement

3
Who? What do you want to know?

-Level 3

Trainer Self ability and competency in :

-training performance

- training scope expertise

- delivering effectiveness level

-work of schedule

-budget of time

- knowledge and skills receive by the trainees

-Level 1 & 2

Human Resources Current and future strategy / planning :

-data to grooming the competency

-Competencies gap

-customer satisfaction

- resolving complaints

-Level 4 & 6

Learners Training objective achievement :

- benefit of the training

-knowlwdge and skills gain from the trainer

-Level 1 & 2

4
Question 3

3. The concepts of ‘return on investment’ and ‘return on expectation’ and a brief

explanation of how these can be measured

Answer

Return on Expectations (ROE)

When designing training programs, it must be taken to the point that expectations of the
training contribution to an overall business initiative are completely clear.

Learning professionals must ask the stakeholders questions to clarify and refine their
expectations on all four Kirkpatrick levels. This is a negotiation process in which the training
professional makes sure that the expectations are satisfying to the stakeholder, and realistic to
achieve with the resources available.

Once stakeholder expectations are clear, learning professionals then need to convert those
typically general wants into observable, measurable success outcomes by asking the question,
“What will success look like to you?” Those outcomes then become the Level 4 Results; the
targets to which you can sharply focus your collective efforts to accomplish return on
expectations.

Return On Investment (ROI)

Cost Benefit Analysis

Cost benefit is a process of determining the economic benefits of a training program. It a

method of assessing the results versus cost incurred.

Why cost ?

• To better understand total expenditure

• To compare cost with alternative training

• To justify the investment

• To evaluate the proportion of money invested on various categories of people

• To control cost
5
 • Improve productivity

• To improve decision making on people development issues

Level Questions Response

1 How many years this hotel has run? 30 years

Do you think this hotel has a beautiful Yes, it is.

landscape?

Is the swimming pool side satisfying Yes, it is.

you?

2 How many staff working here? 300 persons

How many rooms this hotel provide? 250 rooms

How many Meeting Room can be 5 Meeting rooms

rented in this hotel?

3 What is the major problem this hotel Services and attitude problem.
have now?

Refer to the feedback form report,

why did the hotel’s café had many Because the Chief Chef was absent that day.
complaints on 31st August, 2011

Why most customer stayed at our Because the chamber maid supervisor that in
Deluxe Room keep complaining about charge the Deluxe Room had play disappointed
the toilet hygiene? role and action had been taken.

4 Is this business running very well this Yes, with profit margin.
year?

What do you think of developing

another branch of this hotel? It’s good, because this hotel name has establish.

Based on the feedback and Most of the customers satisfied with the services
complaints report, what is your only sometimes when emergency happened,
opinion on customer satisfaction? they complaints on the hygiene services.

6
 Question 4

4. 3 different approaches (methods) to evaluating learning & development activities.

Answer

 Level 1 Evaluation – Reactions Level one serves to be the gauge - it evaluates how
participants/trainees react to the training program or learning experience. It tests
waters by attempting to understand participants' perceptions - Did they like the training
program? Was the training material relevant? Was the method of delivery effective?
The reaction evaluation tools and methods used in this stage are feedback forms,
post-training surveys, questionnaires, which are quick and easy to gather, and non-
expensive to analyse. Often called a smilesheet, this type of evaluation, according to
Kirkpatrick, should be an inherent feature of every training program at the very first
level, for it offers ways in which a training program can be improved. Secondly it builds
the base for level two, as the participants' reactions serve as a pointer as to whether
learning is possible. Even though a positive reaction does not in effect guarantee
learning, a negative reaction to the training program reduces its chances significantly.

- Training Evaluation Form

The advantages and disadvantages of Level 1 Evaluation

Advantages Disadvantages

Quick and easy Not accurate because it is only self

satisfaction.
Low cost
Some respondents not interested to
Can be distributed to big group answer.
Easy to collect data Misunderstanding of the questions.

Not reliable to Attitude-Skill-Knowledge

ASK changes.

7
 Level 2 Evaluation – Learning Level two measures the increase in knowledge - before
and after the training program. In order to do this, tests are conducted on participants
before training (pre test) and after training (post test).

At this stage, evaluation moves beyond participants’ reactions to the newly acquired
knowledge, skills, and attitude of the learners if any. What is important to note is that
this stage does not merely verify skills/knowledge learnt but the extent to which
participants have advanced with regards to new knowledge.

This stage calls for more rigorous procedures, ranging from formal to informal testing to
team assessment and self-assessment. The most common learning evaluation tools
are assessments or tests conducted before and after the training. Interviews,
observation are also not uncommon as they are simple to set up and specific.

(measuring tools : pen & paper test, observation, work sample, assignment/ project,
oral test, checklist, role play/ teach back, Lickert Test/ Attitude Test)

Advantages Disadvantages

(Level 2 of Evaluation : Learning)

Can gain new ASK Not visualize the real work presentation.

Mistakes can be corrected on the spot Time consuming

Easy to set up evaluation tools Cost consuming

Avoid intrepetation and misleading of Need experts to executing testing

understanding.
Need high skill administration
management

-True / False Question, Multiple aswers question, short answer question

8
 Level 3 Evaluation - Transfer The third level assesses the change that has occurred
in participants' behavior due to the training program. At this stage, all evaluation
focuses on the core question - Are the newly acquired skills, knowledge, or attitude
being used by the learners in their everyday work arena?

Did the trainees use the relevant skills and knowledge? Was there significant and
measurable change in performance of the trainees when back to their jobs? Was the
transfer in behaviour retained? Would the trainee successfully be able to transfer
knowledge to someone else? Several trainers view this level as the most accurate
assessment of a training program's success.

However, this stage throws up questions like when, how often, and how to evaluate as
it is nearly impossible to predict when learners will exhibit their newly acquired skills and
behaviour. Hence, during level three, observation and interview over a period of time
are required to measure change, its relevance, and sustainability. Arbitrary, subjective
assessments are unreliable as people change differently at different times. Evaluation
in this area is challenging and is possible only through support and involvement of both
line managers and trainees.

Advantages Disadvantages

(Level 3 of Evaluation : Behaviour)

More accurate result Needs expertise to run testing

Detail in assessment Needs high administration management

More structured test Halo Effect (We assume people are

good)

Lead bias to the assessor

Only suitable for the small group

Difficult to prepare the assessment

9
 Question 5

5. 3 different evaluation tools (e.g. questionnaire, survey,assessment) and their relative

merits.

Answer

 Level 1 Evaluation – Reactions Level one serves to be the gauge - it evaluates how
participants/trainees react to the training program or learning experience. It tests
waters by attempting to understand participants' perceptions - Did they like the training
program? Was the training material relevant? Was the method of delivery effective?
The reaction evaluation tools and methods used in this stage are feedback forms,
post-training surveys, questionnaires, which are quick and easy to gather, and non-
expensive to analyse.

Often called a smilesheet, this type of evaluation, according to Kirkpatrick, should be

an inherent feature of every training program at the very first level, for it offers ways in
which a training program can be improved. Secondly it builds the base for level two, as
the participants' reactions serve as a pointer as to whether learning is possible. Even
though a positive reaction does not in effect guarantee learning, a negative reaction to
the training program reduces its chances significantly.

TRAINING EVALUATION FORM

Course :

Date :

Organizer :

Please tick on the number according to the scale provided below:

1 2 3 4 5

Very Poor Poor Satisfied Excellent Very Excellent

10
Q1. Was the course objective achieved?

1 2 3 4 5

Q2. How do you rate the training’s content?

1 2 3 4 5

Q3. How do you rate the delivery method of this training?

1 2 3 4 5

Q4. How do you rate the trainer’s ability in conducting this course?

1 2 3 4 5

Q5. How the learning environment fulfill your requirement?

1 2 3 4 5

The advantages and disadvantages of Level 1 Evaluation

Advantages Disadvantages

Quick and easy Not accurate because it is only self

satisfaction.
Low cost
Some respondents not interested to
Can be distributed to big group answer.
Easy to collect data Misunderstanding of the questions.

Not reliable to Attitude-Skill-Knowledge

ASK changes.

11
 Level 2 Evaluation – Learning Level two measures the increase in knowledge - before
and after the training program. In order to do this, tests are conducted on participants
before training (pre test) and after training (post test).

At this stage, evaluation moves beyond participants’ reactions to the newly acquired
knowledge, skills, and attitude of the learners if any. What is important to note is that
this stage does not merely verify skills/knowledge learnt but the extent to which
participants have advanced with regards to new knowledge.

This stage calls for more rigorous procedures, ranging from formal to informal testing to
team assessment and self-assessment.

The most common learning evaluation tools are assessments or tests conducted before
and after the training. Interviews, observation are also not uncommon as they are
simple to set up and specific.

(measuring tools : pen & paper test, observation, work sample, assignment/ project,
oral test, checklist, role play/ teach back, Lickert Test/ Attitude Test)

Advantages Disadvantages

(Level 2 of Evaluation : Learning)

Can gain new ASK Not visualize the real work presentation.

Mistakes can be corrected on the spot Time consuming

Easy to set up evaluation tools Cost consuming

Avoid intrepetation and misleading of Need experts to executing testing

understanding.
Need high skill administration
management

12
Question (Test)

True / False

1. Do teamwork is individual work with high performance? (False)

2. Do teamwork need high level understanding & cooperating among team members?
(True)

3. Do teamwork involve more than 10 members? (False)

Advantages (true/ false questions) Disadvantages

Can test a large body of material Difficult to construct questions that are
definitely true or false.
Easy to score
Prone to guessing.

Multiple answers questions

1. What are the values needed in a teamwork?

a. Cooperation

b. Trust & support each other

c. Good team spirit

d. Distrust each other

2. Which is NOT TRUE about quality of teamwork?

a. Respect others

b. Leader centered

c. Mutual cooperation

d. Consensus

13
3. What is the best statement explaining the teamwork?

a. Job done by many people

b. Combined individual tasks

c. Job done with cooperation of many people.

d. Team that is not working

Advantages(multiple choice Disadvantages

questions)

Measures a variety of levels of learning Difficult to construct effective questions

that measure higher order thinking and
Easy to score contain a number of plausible
Can be analyzed to yield a variety of detractors.
statistics

When well constructed, has proven to

be an effective assessment tool

Short answer questions

1. Teamwork is a work performed by a group that contributes towards a mutual goal.

2. Teamwork can improve performance.

3. Teamwork means more we and less I.

4. Teamwork makes difficult works becomes easy.

5. The process of working collaborating with a group of people in order to achieve a goal
is called teamwork.

14
 Advantages Disadvantages

(short answer questions)

Relatively easy to construct Primarily used for lower levels of

thinking
Can cover a wide range of content
Prone to ambiguity
Reduces guessing
Must be constructed carefully carefully
so as not to provide too many clues to
the correct answer.

Scoring is dependent on the judgement

of the evaluator.

 Level 3 Evaluation - Transfer The third level assesses the change that has occurred
in participants' behavior due to the training program. At this stage, all evaluation
focuses on the core question - Are the newly acquired skills, knowledge, or attitude
being used by the learners in their everyday work arena?

Did the trainees use the relevant skills and knowledge? Was there significant and
measurable change in performance of the trainees when back to their jobs? Was the
transfer in behaviour retained? Would the trainee successfully be able to transfer
knowledge to someone else? Several trainers view this level as the most accurate
assessment of a training program's success.

However, this stage throws up questions like when, how often, and how to evaluate as
it is nearly impossible to predict when learners will exhibit their newly acquired skills and
behaviour. Hence, during level three, observation and interview over a period of time
are required to measure change, its relevance, and sustainability. Arbitrary, subjective
assessments are unreliable as people change differently at different times.

Evaluation in this area is challenging and is possible only through support and
involvement of both line managers and trainees.

15
PROCESS CHECKLIST CHANGING A LIGHT
BULB

SUBTASKS WEIGHT 0 1 2 3 MARKS

PROCESS

1. Identify the not working lamp. 1 /3

2. Turn off the switch. 3 /9

3. Let the bulb cool down >30 2 /6

seconds.

4. Wear cotton glove/ dry hand. 3 /9

5. Remove the burnt out lamp 2 /6

6. Dispose the burnt out lamp 2 /6

properly.

7. Fit the new bulb. 3 /9

8. Turn on the light switch. 1 /3

ATTITUDE

1. Tidy and neat. 1 /3

2. Attention ti detail. 1 /3

SAFETY

1. Make sure don’t get shocked. 3 /9

2. Wear cotton glove/ bulb capacity 2 /6

identification

** Weightatage 3 meanscritical area

/72

TOTAL MARKS

PERCENTAGE %

**Min passing marks : 80%

16
PROCESS CHECKLIST CHANGING A LIGHT
BULB

SUBTASKS WEIGHT 0 1 2 3 MARKS

RESULTS Pass: Fail :

Activity 2

Undetake evaluation of an L&D activity, using 2 evaluation tools you have designed yourself,
and analyse your findings. Devise an action plan for implementing at least 2 improvements to
the L&D activity, based on your evaluation, and agree this with relevant stakeholders,

Answer

First Evaluation Tools

 Construct test question

 Distribute to learner

 Mark and score the test papers

 Propose recommendation

Construct 10 questions

True / False

1. Do teamwork is individual work with high performance? (False)

17
2. Do teamwork need high level understanding & cooperating among team members?
(True)

3. Do teamwork involve more than 10 members? (False)

Multiple answers questions

4. What are the values needed in a teamwork?

a. Cooperation

b. Trust & support each other

c. Good team spirit

d. Distrust each other

5. Which is NOT TRUE about quality of teamwork?

a. Respect others

b. Leader centered

c. Mutual cooperation

d. Consensus

6. What is the best statement explaining the teamwork?

a. Job done by many people

b. Combined individual tasks

c. Job done with cooperation of many people.

d. Team that is not working

18
 Short Asnwer

7. Teamwork is a work performed by a group that contributes towards a mutual goal.

8. Teamwork can improve performance.

9. Teamwork means more we and less I.

10. Teamwork makes difficult works becomes easy.

11. The process of working collaborating with a group of people in order to achieve a goal
is called teamwork.

Program Title : Teamwork Skills Training

Objectives : To improve the working skills in team

What went well : Only 1 trainee passed the test. Average score is 65%

What did not go well : 90% of trainees failed in the training test

Recommendation : Should construct new ADDIE for TNA

All employee need to attend refreshment training

Name Score % Passing Score Variance %

%
Mazuki 80 90 10
Sharifah 70 90 20
Suria 50 90 40
19
Shereen 70 90 20
George 50 90 40
Rodney 40 90 50
Mark 70 90 20
Lim 60 90 30
Azreen 70 90 20
Mawi 90 90 0

median 70
mean 65
mode 70
standard
deviation 15.1
range 50

20
Propose recommendation- survey feedback.

NO. LEVEL QUESTION TOOLS/ WHEN WHO +/-

TECHNIQUE

1. REACTION How did the  Feedback

participants form
react to the
trainings?  Interview

2. LEARNING What is the  Pen paper

change in test
ASK before
and after  Oral test
training?
 Observation

 Checklist

 Work
sample

 Role play

 Teach back

 Attitude test

 Assignment

3. BEHAVIOU To what  Observation

R extend the
learners  Check list
apply what
 Work
they have
sample
laernt?
 Attitude test

 Project/
assignment

21
 Second Evaluation Tools

PROCESS CHECKLIST CHANGING A

LIGHT BULB

SUBTASKS WEIGH 0 1 2 3 MARKS

T

PROCESS

9. Identify the not working lamp. 1 /3

10. Turn off the switch. 3 /9

11. Let the bulb cool down >30 seconds. 2 /6

12. Wear cotton glove/ dry hand. 3 /9

13. Remove the burnt out lamp 2 /6

14. Dispose the burnt out lamp properly. 2 /6

15. Fit the new bulb. 3 /9

16. Turn on the light switch. 1 /3

ATTITUDE

3. Tidy and neat. 1 /3

4. Attention ti detail. 1 /3

SAFETY

4. Make sure don’t get shocked. 3 /9

5. Wear cotton glove/ bulb capacity 2 /6

identification

** Weightatage 3 meanscritical area

/72

TOTAL MARKS

PERCENTAGE %

**Min passing marks : 80%

22
PROCESS CHECKLIST CHANGING A
LIGHT BULB

SUBTASKS WEIGH 0 1 2 3 MARKS

T

RESULTS Pass: Fail :

Name Score % Passing Score % Variance %

Mazuki 80 80 0

Sharifah 70 80 10

Suria 40 80 40

Shereen 60 80 20

George 50 80 30

Rodney 90 80 +10

Mark 80 80 0

Lim 70 80 10

Azreen 90 80 +10

Mawi 80 80 0

median 75
mean 71
mode 80

23
standard deviation 16.6
range 50

Program Title : Perfomance evaluation on Changing A Light Bulb

Objectives : To evaluate performance of replacing the faulty bulb safely

Www : Average score is 71%,

W did not go well : 50% of trainees failed in the skills assesment

Recommendation : Revise passing score

Review critical area

Focus re-training to the failure group

Create non training program for failure group ( mentoring and

coaching ), clear instruction, redesign checklist, skills demo by

intructor, mentor mentee ( past guide failure ), retrain the failure

24
THURSTONE SCALES ( TEAMWORK )

1. I will always give the best support to my team.

2. I will always listen to all my team members suggestion.

3. I will always motivate other team members to carrying out the task.

LIKERT SCALES ( TEAMWORK )

Scale :

Strongly agree

Agree

Uncertain

Disagree

Strongly disagree

1. I will always give the best support to my team.

2. I will always listen to all my team members suggestion.

3. I will always motivate other team members to carrying out the task.

Skill and Willl

 Leadership

 Benefit

 Self confident

 Environment

25
Impact

 Cost savings

 Increased output

 Time saving

 Quality improvements

26