Victoria Griffith's Research Methods Project Plan

Simulation training: the opinions of civil construction trainers

external link: cover sheet 200912590

Research Topic Area

A civil construction training organisation in South Australia has recently
purchased a new suite of training simulators. Types of simulators range from
non-motion based, through to six degrees of freedom, but all replicate
construction vehicles such as excavators and mining trucks. The plant trainers
currently use real vehicles for training, but this is expensive in regards to fuel
and supervision ratios. However, this is the way they learnt to operate the
machinery themselves and is the way they have always taught others. As the
organisation looks to formally implement the simulators into plant training, the
researcher (a manager at the organisation) is interested to understand the
opinions of the trainers who will be expected to use them. It is the expectation
that this small scale research will identify the underlying feelings about
simulators in training at the organisation as a means to support the larger scale
change process which will have to occur to optimise their use.

The Literature Review

The literature has confirmed the need to focus on the issue of the research topic
and not to become distracted by the wider discussion points in either simulation
or implementation. The literature review has highlighted some of the differences
in the use of the term simulation which will need to be clarified before the
research can take place. Tichon and Diver (2010) report on the use of simulators
in construction training in Australia, however this study focuses on the impact on
training, not the trainers. Most research in the area of simulation is in relation to
the medical profession rather than the construction industry, but is still relevant
for consideration, raising underpinning theories which will give rise to questions
to put to the trainers, such as whether simulation should be an isolated or
integrated activity.

external link: literature map for research methods

Research Purpose, Problem and Worldview

A federal education improvement grant of over $8 million has funded the
installation of a range of educational technologies at the training organisation
concerned, demonstrating the Australian Governments commitment to

improving education through the use of technology. There is a pressure on the

organisation to prove the successful implementation of the technologies to
enhance teaching and learning in its field, financial sustainability and acceptance
by the industry.

Furthermore, the role of simulation in training has risen in prominence in

Australia over the last decade with the emergence of Simulation Australia, a
national body for those working with simulation. The growth of this organisation
indicates an increase in the interest and use of simulators across a range of
industries and therefore a need to research the components of their
implementation so others can learn from them.

The purpose of this case study is to explore the opinions of trainers at Civil Train
SA, a civil construction training organisation in South Australia, regarding the
implementation of simulators into the courses they deliver. At this stage in the
research, the trainers train plant operators through classroom theory, actual time
on the plant vehicles (about 100 hours) and then a final assessment (theory and
practical). With the scaling back of live plant training (the organisation is moving
the live works site and reducing the amount of live plant available), there will be
an expectation from the organisation for the trainers to use simulators instead.
This qualitative study will seek subjective responses through the use of semistructured individual interviews, to learn the opinions of trainers on the
integration of simulators and to identify any areas for further work to support the
change process. Ultimately, what are the factors trainers consider important in
relation to simulator implementation, both positive and negative, and how are
these likely to impact upon their implementation?

This study will reflect a humanist philosophy; based on the view that truth is a
social construct (one cultures truth may not be anothers) and the value of
human experience. It will seek to explore the perceptions of the team of trainers
so as to facilitate the larger change process at the organisation. The
phenomenology school of thought is based on the meanings we give to the
things we experience and this study will explore how the trainers experiences
with simulators (or lack of) has affected their views and thus the integration of
the technology. Describing and interpreting these experiences and opinions
through this study will facilitate the organisation's ability to change its processes.

Research Question
What are the opinions of trainers at Civil Train SA regarding the implementation
of simulators into plant training?

1. To what extent does previous experience of the simulators affect opinion?

2. Do the trainers feel simulators offer sufficient real to life experiences to be

implemented into plant training?

3. At what stage of plant training would the trainers expect the simulators to
have most impact on training outcomes?

4. What are the concerns of the trainers when planning delivery on the
simulators?

Research Method
The positive and negative aspects of different research approaches will be
subjective and dependent on the philosophies and worldviews of the researcher
and the nature of the research problem. Although there may be assumptions as
to why trainers have not been proactive in integrating the simulators, this study
does not seek to find one truth; the researcher is open to the multiple feelings
and considerations of the participants in the study. There are no expectations to
find one answer to the research problem, but a range of feelings and
interpretations and multiple relationships to be explained. According to Newby
the role of the qualitative researcher is to understand how individuals see the
world(2010, p.119); this study seeks to see how trainers see the world of plant
training with simulators.

Whereas quantitative research is concerned with objective, numerical data,

qualitative data is concerned with subjective responses and the contexts in
which these emerge. Qualitative researchers criticise quantitative data for being
too static and disregarding of the dynamic nature of human data. Qualitative
research may be regarded less reliable because it is based upon perceptions and
personal interpretations of events. Qualitative data is obtained directly from the
participant and may include fact and opinion (Newby, 2010), however while this
type of data is accurate it is not necessarily stable (Newby, 2010, p.143) as
opinions and views can change. However, the use of primary sources makes the
validity and accuracy of the findings in qualitative data high (Colquhoun, 2006)
but as these feelings and opinions can change with time and context, the results
drafted from qualitative research may not be considered reliable as the results
cannot be regenerated if retested on a different occasion. For the qualitative
researcher this is however the objective, as the method will have been chosen

because quantitative research could not measure such aspects. Newby notes
that the flexibility afforded by qualitative approaches means that there must be
researcher excellence (Newby, 2010, p.115). As a novice researcher, this
concerning, however this will be minimised by thoroughly investigating the
reliability and validity of the proposed research.

Qualitative approaches are naturalistic, meaning data is collected in as natural a

setting as possible. Whereas the qualitative researcher is immersed in the
research context, the quantitative researcher is objective and uses non-human
instruments (Colquhoun, 2006) for data collection and analysis. Data will be
collected from the trainers in their day to day working setting, asking them about
instances which are part of their roles. However, it is often difficult to be truly
naturalistic because the very nature of collecting research forms an unrealistic
situation because the researcher is known to the subjects as their manager.
Nonetheless, it is common for the researcher to conduct a study in their own
institution in education research due to limited time and funding (Conklin, 2010).

Qualitative data are the words and meanings by which individuals express and
give meaning to their unique experience (Colquhoun, 2006). As there is no
hypothesis to be tested, the theory emerges and develops from the data.
Qualitative research is considered as less structured which may be seen as a
negative aspect for some who do not seek to deal with the unpredictability of
human emotions. However, this adds flexibility which qualitative researchers see
as a positive aspect for those addressing their research from a humanistic
perspective. Qualitative approaches will be used to discover the connections
between social, emotional, economical or cognitive circumstances as to whether
simulators can be implemented into the training programmes. A strength of
qualitative research methods is the way it aims to sees the whole picture and
combine different sources, however quantitative researchers may consider this
to be a weakness as there is no one aspect to make a judgement against
(Newby, 2010, p.117).

Qualitative research approaches are suitable for this study as probabilities do

not tell us what we have to do to change the situation (Newby 2010, p.117). This
research is not looking to find out how many trainers do or do not think the
simulators should be involved in training, but the reasons why simulators should
or should not be used and how they feel is the most suitable way to start
integrating them into the training they deliver.

The methods of data collection relevant to the qualitative approach taken in this
study could include interviews, questionnaires, focus groups and observation. If
the study was dealing with a large sample where there was a need to collect lots

of data in a structured way, then questionnaires would be a suitable collection

method (Newby, 2010). However, interviews offer more flexibility in that if a
question is misunderstood or a new point of interest arises, the interviewer can
choose to explore it further. Therefore, for this study, semi-structured individual
interviews will be used. This style of interviewing is considered constructionist as
the role of both interviewer and interviewee are active (Roulston, 2011). This
method would be suitable to administer given the small size of the sample. Some
interviews may have to be undertaken on the telephone where trainers are
working regionally. Where permission is granted I will record face to face and
telephone interviews, although I recognise that where telephone interviews are
recorded responses may be more guarded (Newby, 2010, p.340). An interview
guide with starter questions will be developed, with guidance on interviewer
behaviour. Due to the small scale nature of this study, the researcher will be
acting as the sole interviewer so there will not be any training time or resources
required. Interviews will be conducted in the space of two days so as to minimise
latter interviewees being influenced by any contextual or social factors.

An authoritative source has knowledge of the issue and credibility as a source.

Authoritative sources for this study are the trainers at the organisation and peer
referenced articles on the use of simulators. Trainers are credible sources
because their role is central to the study and the implementation of simulators,
and they have a wealth of experience in plant operations and vocational training.
Newby states that a source can be credible when providing opinion as data if
they have the power to change situations (2010, p.224). The trainers have the
power to successfully change the rate and success of simulator use. According to
Newby (2010, p.225), an authoritative source should be stable over time.
However, qualitative data is not stable, as identified by Newby himself (2010,
p.143).

The target population is determined by the research issue which in this study is
the implementation of new educational technologies to maximise training
outcomes in civil construction training (full time and contractor civil construction
and plant trainers at Civil Train SA). Thus, the target population is all trainers at
Civil Train SA.

The sampling frame is all trainers at Civil Train SA; the same as the population.
This is the most accurate sample because we can list every element in the
population (Newby, 2010, p.232). The sample to be used is non EPSEM (Non
probability of selection method or purposive) case study as it is the study of an
individual situation. If this study is an exemplar of an organisation adopting new
educational technologies then it may be representative and therefore constitute
a sample (Newby, 2010, p.253). As one branch of a national organisation, Civil
Train SA is leading the way in the implementation of educational technologies

and will serve as the blueprint for wider roll out, and therefore the trainers
identified for data collection in this study would be representative of civil
construction trainers across the state branches.

Data is raw stuff (Newby, 2010, p.141) and the role of the researcher is to
extract information from the data. The data collected from this study will be in
the form of the words, and analysed as the words themselves and the emphasis
they are given (Newby, 2010, p.454). It is expected that the process of data
analysis will contribute to the understanding of the research issue and will be
accepted by the professional audience. Although findings will not be published in
a journal or report, they may be used as part of a larger scale internal research
project in the future.

Coding is a form of data analysis, the object of which is to name units of data
(Newby 2010, p.462). Template analysis is one option where codes are applied to
the data from the top down (Newby, 2010, p. 481). The template could be used
to structure the interview questions, however this study expects to see
relationships emerge from the data and therefore template analysis is
unsuitable. Qualitative content analysis creates a coding structure based on the
researchers interpretation and identification of meaning and therefore appears
more appropriate to this study. There are methods suggested for handling
interview data, such as that used by Fereday and Muir-Cochrane (as cited in
Newby, 2010, p.483). However, analysis through coding may extend above the
needs of this small scale research study as with no clear boundaries as to when
to stop, the time demands and pressures on the researcher can sometimes
outweigh the benefits (Newby, 2010, pg. 465).

Phenomenography may be used in data analysis as this method is closely linked

to the starting position (Newby, 2010, p.509) of the study. This is appropriate
because this research study seeks to understand how the trainers perceive the
world in which they operate (Newby, 2010, p.509); the world being the
emerging use of educational technologies in plant training. Although there are no
specific methods for this type of analysis, a key element is the need to
understand what the subject is saying and be sure that the subject is saying
what they are actually experiencing (Newby, 2010, p.509). This requirement is
satisfied by the insider knowledge of the researcher into the context. Perception
or experience of the phenomenon and the understanding of the phenomenon are
different (Dahlin, 2007, p.332) and must be considered so in preparation for both
the data collection and data analysis. When preparing interview questions, the
focus of this study will be to seek perceptions of the trainers as to simulation and
consider this in relation to their understanding of simulators. It may be that the
one is identified as impacting on the other.

Ethics, Reliability and Validity

Validity determines that the approaches and techniques used to collect the
research are appropriate. This study uses methods and approaches according to
the needs identified by the research problem, hence the researcher is immersed
into the research context. However, in using this approach, validity may be
questioned. Triangulation or convergent validity (Cohen, Manion and Morrison,
2007, p.150) is a method used to validate the outcome by at least two sources.
In this study, interview responses could be corroborated with documentary
evidence, however, as seen by the literature review, there is limited relevant
research to use. Responses from the individual subjects could be compared to
see if they corroborate each other, however, the fact that this study deals with
opinions and feelings within a small pool may mean this will not happen.
Minimising bias is a way to ensure validity in qualitative research (Cohen, Manion
and Morrison, 2007, p.150), however given the interpersonal interactions of
interviews this can be difficult to achieve. The researcher could use someone
else as the interviewer to mitigate bias because of the researcher's position of
authority (Hume and Young-Loveridge, 2011), however, the resources are not
available to hire an associate and a colleague could place additional burden on
the organisation. As there is only one interviewer, there is no need for
interviewer training and there is not the time to schedule pilot interviews in this
study. Where this study uses telephone interviews in lieu of face to face
interviews to enable access to trainers working regionally, validity may be
questioned because of the lack of visual gestures (Cohen, Manion and Morrison,
2007, p.153). This method will be used as a last resort where face to face
interviews are not possible and where it would be detrimental to the study not to
interview a participant rather than interview them over the telephone. Although
online data collection methods may be considered appropriate for gathering
responses from geographically distanced subjects, they will not be used for this
study (other than initial contact requesting participant consent). However, using
the internet to gather qualitative research data (through the use of online forums
or online questionnaires for example) could be seen as raising issues around
ethics and reliability (Crompvoets, 2010).

For the results of this study to be considered reliable, they must be able to be
replicated. Highly structured interviews would increase reliability, however this
method of data collection did not support the flexibility required by this study.
Variations in wording may change the tone of the question and therefore the
response (Oppenheim 1992 cited in Cohen, Manion and Morrsion, 2007, p.150).
However, it is also argued that the uniqueness and idiosyncrasies (Cohen,
Manion and Morrison, 2007, p.148) of qualitative responses is their strength, not
a weakness (Silverman, 1993 cited in Cohen, Manion and Morrison, 2007, p.151).
Therefore, by using semi structured interviews, this study may offer balance
between structure and flexibility. There are some authors who have come up with

methods to test the reliability of qualitative study such as Denzin and Lincoln
(1994 cited in Cohen, Manion and Morrison, 2007, p.148). However as Cohen,
Manion and Morrison (2007) point out, the differences between the reliability of
qualitative and quantitative research is not justified considering the choice of
approach is about fitness for purpose and therefore the approach will be selected
because of these characteristics, not in spite of them.

Reliability in qualitative research is about fidelity (Cohen, Manion and Morrison,

2007, p.149). Given the organisational hierarchy difference between interviewer
(Manager) and interviewees (trainers), there may be an issue around power.
However, it could be argued that the interview process provides power to the
interviewee as they have the control over the response they provide (Cohen,
Manion and Morrsion, 2007, p.152). Although in this study, the researcher is the
participants Manager, there is not expected to be an issue of acquiescence
(Cohen, Manion and Morrison, 2007, p.151) because the contextual knowledge of
the researcher should highlight such issues. Hence, the role of the interviewer is
vital in semi structured interviews in ensuring the validity of the knowledge
produced (Hume and Young-Loveridge, 2011, p.114). In semi structured
interviews many of the methodological decisions will be made on the spot to deal
with the flow of the interview, but this skill is comparative to the knowledge of
the interviewer knowing what they are asking about and why they are asking
(Hume and Young-Loveridge, 2011, p.115). Therefore, immersion of the
interviewer/researcher into the context is important for this method of data
collection to be successful. However, Drake (2010) identifies some of the issues
that can occur when researching from the inside, including authenticity of data
collection. Insider interviews in particular can be problematic due to the shared
knowledge of the workplace and the interviewee may see themselves as coproducer of knowledge rather than respondent providing raw data (Drake, 2010,
p.86).

The results must be valid to be accepted (Newby, 2010) and therefore the
methods of analysis are just as important as methods for data collection. There
could be an issue of transcriber selectivity (Cohen, Manion and Morrison, 2007,
p.154), where responses are interpreted and this may affect reliability.

Ethics have been considered at each stage of this research plan. There are no
ethical issues that arise from the nature of the research issue itself. There may
be an ethical issue as to the data collection methods due to the relationship of
the researcher and the subjects. However, as the research is a qualitative study
reflecting a humanist philosophy, the subjective approach of the researcher is
expected. Ethical reflexivity (Naidu and Sliep, 2011, p.434) will be applied
throughout the study and not just once ethical approval has been sought. It is
the aim of the researcher to apply ethics in practice not just procedurally.

Consent will be collected from the participants although it is noted this is not
always necessary (Cohen, Manion and Morrison, 2007, p.53) and this is not a
study where participants will be exposed to any stress, pain, invasion of privacy
(Cohen, Manion and Morrison, 2007, p.52). Consent will be informed so
participants (all over 18 years old) are fully aware of their involvement. It will be
made clear to participants that this research is independent of their employment
at the training organisation and individual responses will not be identified in the
write up via external link: Participant letter and external link: Participant
consent. As all participants are on the same authority level, there should be no
tell-tale signs as to who has provided responses through the final report (Newby,
2010). This is a one off study, where participants will be needed for a small
amount of time, so access should not be a problem for this research, unless there
are unforeseen circumstances such as staff sickness or annual leave.
Furthermore, the researcher is an insider (Cohen, Manion and Morrison, 2007,
p.56), although access should not be taken for granted. Formal approval
(external link: organisation consent) has been sought from the organisation to
undertake the study within the proposed professional context, so the
organisation is clear as to the demands the study will place on their staff.

An external link: Ethics approval form has been completed for this study and
approved by the Faculty Ethics Committee on 30 May 2012.

Results
Semi structured interviews were conducted using an external link: Interview
template over a period of two weeks. Four interviews were completed, three face
to face, one over the telephone. Interviews lasted between twenty and forty
minutes. The interviews were conducted to allow the participant to control the
content of the conversation, but with the interviewer using the interview
template to ensure all points were addressed. Interviews were not recorded.
Notes were scribed by the interviewer during the discussions.

Although the number of interviews completed was less than planned, each
interview was in depth and included more raw data than originally expected.

It was difficult to find time to speak to trainers face to face that did not interfere
with their day to day jobs. Also, my role as Manager put constraints on when I, as
the interviewer, could be available for interviews. As expected, regional training
demands did impede face to face interviews. Furthermore there was an
unexpected resignation of two permanent staff reducing the pool of interviews

further. Contractor trainers have been less inclined to volunteer to participate, I

assume because they have less loyalty to the organisation than an employee.
Given another two to three weeks I could have probably managed to get around
another four or five trainers, however, the scope of this assignment did not allow
for this extension of scope (although I will endeavor to continue to interview staff
to provide feedback to the organisation and to help with the expected
organisational change).

Once interviews were completed, the interviewer drew notes, phrases and
descriptions into one document external link: collation of results

References
Cohen, L., Manion L., and Morrison, K., 2007. Research methods in education
(ebook). 6th ed. London: Routledge. Available through: Dawsonera website
www.dawsonera.com (Accessed 1 June 2012).

Colquhoun, D. 2006. Research methods in education. Institute for Learning.

University of Hull.

Conklin, H., 2010. Compassion and mindfulness in research among colleagues.

Teaching Education, (e-journal) 20(2), pp. 111-124. Available through: Ebscohost
database external link: http://web.ebscohost.com/ehost/pdfviewer/pdfviewer?
vid=6&hid=12&sid=340e7594-070c-4cad-b2b2-2b94489a617c%40sessionmgr4
(Accessed 2 June 2012).

Crompvoets, S. 2010. Using online qualitative research methods in medical

education. International Journal of Multiple Research Approaches, (e-journal) 4
pp.206213. Available through: Ebscohost database external link:
http://web.ebscohost.com/ehost/pdfviewer/pdfviewer?sid=3f6efb6a-29be-4c4daf67-d4b373b6656d%40sessionmgr14&vid=6&hid=9 (Accessed 30 May 2012).

Dahlin, B., 2007. Enriching the theoretical horizons of phenomenography,

variation theory and learning studies. Scandinavian Journal of Educational
Research, (e-journal) 51(4), pp. 327-346. Available through: Ebscohost database
external link: http://web.ebscohost.com/ehost/pdfviewer/pdfviewer?
sid=75810823-33e6-4d87-b8bc-c385a41c9aa7%40sessionmgr4&vid=4&hid=7
(Accessed 1 June 2012).

Drake, P., 2010. Grasping at methodological understanding: a cautionary tale

from insider research. International Journal of Research and Method in Education,
(e-journal) 33(1), pp.8599. Available through: Ebscohost database external link:
http://web.ebscohost.com/ehost/pdfviewer/pdfviewer?
vid=4&hid=12&sid=340e7594-070c-4cad-b2b2-2b94489a617c%40sessionmgr4
(Accessed 1 June 2012).

Hume, A. and Young-Loveridge, J., 2011. Using professional colleagues as

interviewers in action research: possibilities and pitfalls. Waikato Journal of
Education, (e-journal) 16(3), pp. 111-124 Available through: Ebscohost database
external link: http://web.ebscohost.com/ehost/detail?
vid=5&hid=9&sid=3f6efb6a-29be-4c4d-af67-d4b373b6656d
%40sessionmgr14&bdata=JnNpdGU9ZWhvc3QtbGl2ZQ%3d
%3d#db=ehh&AN=71496142 (Accessed 30 May 2012).

Naidu, T. and Sliep, Y., 2011. Contextual reflexivity: towards contextually relevant
research with South African HIV/AIDS home-based care volunteers. International
Journal of Qualitative Methods, (e-journal) 10(4), pp. 431-443. Available through:
Ebscohost database external link:
http://content.ebscohost.com/pdf27_28/pdf/2011/QRZ/01Dec11/70139587.pdf?
T=P&P=AN&K=70139587&S=R&D=aph&EbscoContent=dGJyMMTo50Sepq84yOv
sOLCmr0qeprNSsKy4TbGWxWXS&ContentCustomer=dGJyMPGttVC3prJOuePfgey
x44Dt6fIA (Accessed 1 June 2012).

Newby, P., 2010. Research methods for education. London: Pearson Education
Ltd.

Roulston, K., 2011. Working through Challenges in Doing Interview Research.

International Journal of Qualitative Research, (e-journal) 10(4), pp. 348-366.
Available through: Ebscohost database external link:
http://web.ebscohost.com/ehost/pdfviewer/pdfviewer?sid=09c2cdfa-ad90-476295be-310d911223a9%40sessionmgr4&vid=10&hid=7 (Accessed 31 May 2012).

Tichon, J. and Diver, P., 2010. Plant operator simulation: benefits and drawbacks
for a construction training organization. Cognition, Technology & Work, 12(3),
pp.219-229.