ACTION RESEARCH

DATA COLLECTION CONSIDERATIONS

NOOR AFIDA AYU BT MAMAT

SITI ZUBAIDAH BINTI PA'AT
WAN NUR SYAHIRAH BINTI WAN
HANAFFI
PREPARED BY : AFIDA AYU, ZUBAIDAH,
SYAHIRAH
 Sampling, validity, reliability,
OVERVIEW bias
Sampling and bias
Validity : - Data triangulation
- Analysis check by research participant

Reliability

Ethics
 SAMPLING
Trochim (2002) defines sampling as the procedure of choosing
components from a desired group of so that by conducting a study on
them we could generalise our findings and apply them on the rest of
the society taken from.

Involves taking representatives selection of the population and using

the data collected as research information.

A sample is a subgroup of a population. (Frey et al. 125)

5 STEPS TO SAMPLING
Identify the population

Specify a sampling frame

Specify a sampling method

Determine the sample size

Implement the plan

 VALIDATING

How we know that the data we collect

(e.g test score) accurately gauge what
we are trying to measure?
 VALIDATING
Refer to the degree to which scientific observation actually
measure/record what they purport to measure (Pelto & Pelto, 1978,
p. 33)

To use multiple sources of evidence as the way to ensure

construct validity. (Yin , 2003)

Using three different data sources (triangulation) such as surveys,

checklist and test scores help to validate the research findings
(makes the research study rigorous)
 RELIABILITY

The degree to which a test consistently measures

whatever it is measuring.

The more reliable a test is the more confidence we can

have that the scores obtained from the test are
essentially the same scores that would be obtained if
the test were re-administrated to the same test takers.
 Tells the user
VAL
about the
IDIT
Y
appropriateness
of a test.

Tells us about
RELI
the consistency ABIL
of the scores ITY
produced.
 BIAS
Bias is present when respondents ethnicity, race, gender,
language or religious orientation distort their performance or
responses.

E.g, low scores on reading tests by students who speak little

English or nonstandard forms of English are probably due in
large part to language disadvantages, not reading difficulties.
 SAMPLING & BIAS
Sampling bias is generally fault of the researcher.

Some aspect of the sampling creates a bias in the data.

For example, when a researcher gets a return of

only 45% of questionnaires sent out, the large
number of no returns introduces a potential
response bias in the results.

The sampling bias in the study makes the study

conclusions invalid.
Data triangulation
Gathering data through one technique can be
questionable, biased and weak.
Researchers triangulate among different sources of data
to enhance accuracy of their study (Creswell 2008).
Increases the credibility of the research by drawing on
multiple viewpoints.
They tap into a variety of sources of information,
confirmation, individuals and processes of data
collection.
 Creswell explains the idea of data
validation by saying that triangulation is
the process of corroborating evidence from:
different individuals such as parents, students
and teachers in a school setting
Therefore, if we obtain the same results,
we can become sure that the data are
valid.
Analysis-check by research
participant
Also known asinformant feedback or
respondent validation.
A technique used by researchers to help
improve the accuracy,
credibility,validity, and applicability,of a
study.
 The interpretation and report is given to
the interviewees in order to confirm the
content and to check the authenticity of
the work.
Their comments serve as a check on the
viability of the interpretation.
In this way the plausibility and truthfulness
of the information can be recognized and
supported.
How to conduct?
Participants will critically analyze the findings and
comment on them.
They will either affirm that the summaries reflect their
views, feelings, and experiences, or that they do not
reflect these experiences.
If the participants affirm the accuracy and
completeness, then the study is said to have credibility.
These checks are not without fault, but serve to
decrease the incidence of incorrect data and the
incorrect interpretation of data.
The overall goal of this process is to provide findings
that are authentic, original and reliable.
 RELIABILITY
Reliability is a different but no less important concept.
Reliability relates to researchers' claims regarding the
accuracy of their data.

To enhance the reliability of your action research data, you

need to continually ask yourself these questions when
planning data collection:
Is this information an accurate representation of reality?
Can I think of any reasons to be suspicious of its accuracy?
Inter-Rater/Observer Reliability: The degree to which different
raters/observers give consistent answers or estimates.

Test-Retest Reliability: The consistency of a measure

evaluated over time.

Parallel-Forms Reliability: The reliability of two tests

constructed the same way, from the same content.

Internal Consistency Reliability: The consistency of results

across items, often measured with Cronbachs Alpha.
ETHICAL ISSUES
Role of the Researcher
BASIC ASSUMPTIONS ABOUT HOW RESEARCH SHOULD
BE CONDUCTED

Subjects should be protected from harm.

Subjects should have their identity protected.

Subjects should be fully informed about the research study.

Participation is voluntary.

Study procedures should show respect for cultural values and

beliefs.
 Some studies can potentially harm participants.
Consequently risks should be minimized. Potential
harm can include:
Pain or physical danger.

Emotional arousal or stress

Observation or release of findings can cause embarrassment

or social distress.

Observation can involve misinformation or deception.

Participant observation techniques should be used cautiously.
Except for observation in public settings,
Participants must give consent to participate. We
are especially concerned about:
Children under 18 (consent must be obtained from
parents).

Involuntary clients

Any participants likely to be vulnerable to coercion or

undue influence.
 COMPONENTS OF INFORMED
CONSENT INCLUDE:
Accurate explanation of evaluation procedures is made.

Respondent is given an explanation of risks and

benefits

An offer is made to answer any additional questions

about the project (contact information must be
included)

The participant is told that he or she can withdraw at

CONSENT LETTER
Steps must be taken to protect
participants from harm:
Confidentiality must be protected. We must not identify
people who participate in the study.

If the study is likely to cause emotional arousal, we

must make provisions to refer participants to a mental
health professional for counseling.
Methods for Protecting
Confidentiality Include:
Responses must be anonymous (as feasible).

All information that can be attributed to individuals is kept confidential.

Fictional names are often used in qualitative analysis or code numbers
are assigned to individuals for conducting the analysis. Respondents may
be referred to using terms that do not indicate name/characteristics of
individuals such as respondent or participant). In small samples,
care should be taken not to reveal much about personal characteristics
of respondents such as ethnicity or job title if it would help readers
identify individual respondents.
A coding system can be used to track returned surveys
or case records. However, the coding system should be
kept in a secured location separate from the responses.
The responses are also kept in a secure location; only
the researcher will have access.
Any instruments that could identify a respondent should
be destroyed after data analysis if it can not be kept in a
secure location. This includes tape recordings.
Information about individual respondents should not be
shared with agencies or supervisors.
Methods used to verify consent to
participate
Return of surveys implies consent (surveys should be
distributed with letters that contains information about the
study and human subjects protections).
Participation in phone interview implies consent.
Signed consent forms from participants (in some cases it
may be sufficient to provide verbal information).
IMPORTANT!!! Consent forms are needed for all research-
related interviews other than interviews with public officials.
No consent forms are needed for brief interviews in public
settings
Public behavior (observations) generally does not require
consent.
Cover letters and consent forms should contain
the following information:
A description of the purpose of the study.

An explanation as to how participants were

selected.

A statement that participation is voluntary and

that participants may withdraw at any time.

A description about procedures and what will be

required from participants.

Emotionally sensitive issues that might be

exposed and/or follow-up resources that are
A description of how information will be recorded if
videotaping or audiotape is required.

A description of any discomforts and any known risks.

An explanation of who will have access to the data

and information about the identity of respondents.

A description of how the data will be made public or

of any other persons who may make use of the data

Contact information (for the researcher or the

researchers institution) if the participant has any
questions.
Additional human subjects issues
Payment for participation (o.k. if everyone is paid).

Case record analysis requires that steps be taken to protect

confidentiality of subjects and people described in the case
record.

Deception. (May be used in some cases where it will not

cause undo harm or when authorities have been given
permission).
Ethical Issues in Qualitative Research
Researchers must state values and biases in writing reports.

Researchers must take steps to ensure that accurate accounts of

participant perceptions are written

Researchers have a responsibility to use the data to enhance social

change.

Consent is sometimes obtained through personal interaction with

individuals or communities (entry). The researcher must establish
trusting relationships.

Participants may be viewed as partners in the research process and

always as the social equal of the researcher.
ESTABLISHING ACCURACY OF DATA IS CALLED
TRUSTWORTHINESS.
Rather than reliability and validity of instruments, in qualitative
research we talk about whether the research is trustworthy.

Padgett defines trustworthy studies as those that are ethical and

fair and that portray respondents experiences and perceptions
accurately.

Assumption is that the researcher is the instrument and that the

observation or interview is filtered through the researcher and the
researcher writes up the findings.
Primary Trustworthiness Concerns
Reactivity (Hawthorne Effect) have people reacted to
being studied.
Researcher biases, own values/perceptions or
becoming too close or two distant from respondents.
Respondent biases may lie or be to helpful
Procedures for establishing
trustworthiness
Obtaining feedback on written results from respondents.
Prolonged engagement (observational studies or interviews
conducted over a long period of time)
Member checking asking respondents for feedback on
data analysis and coding
Triangulation using several data sources (for example, both
interviews and observation) or more than one observer.
Peer debriefing and support
Negative case analysis
Audit trail
Cultural Competency
Check to make sure that subjects can understand concepts described in
survey or interview guide (pre-test). Are terms used that may have
different meanings in different cultures.
Is the level of language appropriate to the participants?
Translations should be checked for accuracy and appropriateness of
language to social class, age, and other demographic characteristics of
participants (for example, formal versus informal Spanish).
Do study procedures violate any cultural norms (check with an informed
source in the community).
Other qualitative issues:
Boundaries (appropriate amounts of involvement vs. social distance)
Keeping records of ones own reactions as part of the research process
dont interject them into the setting though.
Protect confidentiality except in situations involvement immediate harm.
Make sure to take precautions to protect ones self.
Need to be able to establish relationships with respondents (engagement
skills).
Need to be flexible especially during the data collection process and
open to new things and ways of understanding them.
Re: Moral dilemmas Reporting negative
behavior (from Padgett):
Sometimes our respondents are involved in activities
that are not dangerous, but are still illegal, for example,
shoplifting, prostitution, drug use, or truancy from
school. Qualitative researchers cannot allow their
personal or moral concerns about these behaviors to
interfere with the promise of confidentiality (p. 39).
For reporting abuse, Padgett
suggests that the researcher
consider whether

It is necessary or beneficial to report when

data is being collected.
It may be sufficient to use the findings to
engage in social action after the final
report is written.
Thank You..
Happy
Researching!