METODOLOGI PENELITIAN

(Research Methodology)

Suhendro Yusuf I, Ph.D

School of Computer Sciences, Darmajaya Informatics and Business Institute
 References

http://www.csc.liv.ac.uk/research.html
http://www.cs.umd.edu/~vibha/634.html
http://www.ecs.soton.ac.uk/syllabus/elec6021.html
http://www.cs.jhu.edu/~nasmith/erm
Description

The study and application of research methods appropriate to

graduate students studies. The course will provide a general
research methods, as well as providing practical exposure to
problem statements, Literature Reviews, Writing the Research
Proposal, and Organization of the Research
Report. Quantitative and Qualitative Research methodologies
will be briefly covered in preparation for the later courses in
these areas
Aims
- To provide a deep and systematic understanding
of the nature and conduct of CS research.
- To enhance existing transferable key skills.
- To develop high order transferable key skills.
- To equip students with the ability to undertake
independent research.
- To remind students of the Legal, Social, Ethical
and Professional (LSEP) issues applicable to the
computer industry.
Learning outcomes

At the end of the module students will be expected to:

Have an understanding of how established techniques of research

and enquiry are used to extend, create and interpret knowledge in
Computer Science.

Have a conceptual understanding sufficient to: (i) evaluate critically

current research , and (ii) propose possible alternative directions for
further work.

Be able to deal with complex issues at the forefront of the academic

discipline of Computer Science in a manner, based on sound
judgments, that is both systematic and creative; and be able to
communicate conclusions clearly to both specialists and non-
specialists.
Teaching and learning strategies

Formal Lectures and Seminars: Students will be expected to attend

one and half hours of formal lectures and seminars in a typical week.
Formal lectures will be used to introduce students to the concept and
methods covered by the module. These will be supplemented by a
series of presentation in which students will give an overview of their
research, thus introducing students to state-of-the art research
topics and how they are approached.

Private study: In a typical week students will be expected to devote 9

hours of unsupervised time to private study. The time allowed per
week for private study will typically include 2.5 hours of time for
reflection and consideration of lecture material and background
reading.
Rules and regulations:

Mobile phone in silent mode during class session

15 minutes late tolerant

80 % attendence required to participate in Final exam.

University rules and regulations should be obeyed

Assessments

Final Exam (UAS): 40%

MidTerm Exam: 25%

Assigment : 30%

Presence: 5%
WHAT IS RESEARCH
 research?
• what is it?
• should you be doing it?
• how do you do it?
 definitions of research
“Systematic investigation towards increasing the sum of knowledge”
(Chambers 20th Century Dictionary)

“an endeavour to discover new or collate old facts etc. by the scientific
study of a subject or by a course of critical investigation.”
(The Concise Oxford Dictionary)

The good researcher is not ‘one who knows the right answers’
but ‘one who is struggling to find out what the right questions
might be’. (Phillips and Pugh (2005: 48))
 What is research?
• research. 1.a. the systematic investigation
into and study of materials, sources, etc, in
order to establish facts and reach new
conclusions. b. an endeavour to discover
new or collate old facts etc by the scientific
study of a subject or by a course of critical
investigation. [Oxford Concise Dictionary]

12
 What is research?
• Research is what we do when we have a question or
a problem we want to resolve
• We may already think we know the answer to our
question already
• We may think the answer is obvious, common sense
even
• But until we have subjected our problem to rigorous
scientific scrutiny, our 'knowledge' remains little
more than guesswork or at best, intuition.

13
 What is research?
• First priority is to formulate your question
• Then figure out how you are going to answer
it
– How have others answered it?
– How does your proposal fit in with what others
have done?
– How will you know when you have answered it?
• Then you can present your answer

14
 definitions of research
• Research is defined by the Higher Education
Funding Council for England (HECFE) as
‘original investigation undertaken in order to
gain knowledge and understanding’ (RAE,
2008).
• Three key terms in this definition have been
italicized for emphasis; original, gain and
knowledge and understanding.
 Originality
• There is no point in repeating the work of
others and discovering or producing what is
already known
• originality is doing something that has not
been done before. While this remains a
relatively simplistic definition, it is important
to discuss how originality relates to projects.
 Originality
• What can you do that is original?
• What type of things can you produce that are
original?
 Originality
You can be original in two ways.
1. You can be original in the way you do things –for
example, doing something someone has done
before but using a different technique or approach.
2. You can be original by producing or developing
something that has not been produced before.
 Originality
Number of areas in which your project can be original:
1. You can be original in the way you do things –for
example, doing something someone has done
before but using a different techniqueor approach.
2. You can be original by producing or developing
something that has not been produced before.
 Originality
• Identifies a number of areas in which your project
can be original:
– Tools, techniques, procedures and methods. You may apply new
tools and techniques. to existing problems or try new procedures and
methods in contexts where they have not been applied before.
– Exploring the unknown. Although rare, you may investigate a field
that no one has thought to investigate before.
 Originality
• Identifies a number of areas in which your project
can be original:
• Exploring the unanticipated. Although you may investigate a field of
research that has been looked at many times before, you may come across
unexpected results or exciting new directions as yet unexplored.
• The use of data. You can interpret data in different ways, use them in new
ways or apply them in alternative areas that have not yet been
investigated.
 Contribution to knowledge
• Body of knowledge represents world understanding
– Theories
– concepts
– models,
– the sciences,
– the arts and so forth.
• Knowledge is stored in:
– books,journal articles,
– conference proceedings,
– documents,
– reports, the Internet, art, peoples, minds and more.
 Hierarchy of Knowledge
• data,
• information,
• knowledge and
• wisdom.
 Hierarchy of Knowledge
• Data are the factual elements that describe objects or events.
They represent the raw numbers and raw text you gather
from your investigations.
– rainfall data from various sites around the country.
• Information, represents data that have been processed in
order to provide you with some insight into their meaning.
Information provides you with an idea of the ‘what’ (i.e., what
is happening in the
– Converting your rainfall data into information may lead to
graphs summarizing monthly totals, charts presenting
seasonal fluctuations and text or tables summarizing
average daily rainfall at different sites.
 Hierarchy of Knowledge
• Knowledge is your higher-level understanding of things.
Knowledge represents your understanding of the ‘why’.
Knowledge is your personal interpretation of what you gain
from information as rules, patterns, decisions, models, ideas
and so on.
– knowledge represents your understanding of why rainfall might have
changed during this period
• Wsdom.
 Hierarchy of Knowledge
• Wisdom. Wisdom represents your ability to put your
knowledge into practice. It represents your ability to apply
your skills and experiences to create new knowledge and
adapt to different situations.
– Wisdom would represent your ability to predict likely changes to
rainfall and climate in the future or enable you to understand why rain
falls at particular levels in entirely different parts of the world
 Theory
• Data, information,knowledge and wisdom represent
a relatively ‘firm’ understanding of what is going on
and how things can be applied,
• Theory represents ideas, opinions and suppositions
based on your observations of the world. A theory is
not necessarily true but, at the moment, it
represents the best explanation of what you observe
 The research process
• sequential,
• generalised,
• circulatory
• and evolutionary:
 Intellectual discovery
• Inductive reasoning
– You start with your observations of the world and
come to general conclusions about it. In other
words, you build models and theories based on
your interpretation of the world.
Inductive reasoning
You start with your knowledge and understanding of the
world and predict likely observations within it, even though
you might not have encountered them before.
• Inductive reasoning
– You start with your observations of the world and
come to general conclusions about it. In other
words, you build models and theories based on
your interpretation of the world.

All swans I’ve ever seen are white

Inductive conclusion: All swans are white
• Random guesses
• Analogy. Is the problem similar to anything else that already has a solution
or explanation?
• Inversion. Try to look at things from the opposite angle. For example,
instead of asking ‘which computer language should I use?’ ask ‘why
shouldn’t I use Java?’.
• Partition. Break the problem or situation down into smaller, more
manageable and understandable parts
 Classifying research
• Field :The field of research is ‘little more than a labelling device which
enables groups of researchers with similar interests to be identified’

– Example:
• Artificial Intelligence ! Automated Reasoning !
• First-Order Reasoning ! Decidability

• Approach Research methods that are employed as part of the research

process
– Examples:
• Case study, Experiment, Survey, Proof

• Nature
• The type of contribution that research makes to knowledge depends upon
its nature.
 Classifying research
•
• Nature
– Pure theoretical development
– Review of pure theory and evaluation of its applicability
– Applied research
 Classifying research
• Pure theory – developing theories to explain things without necessarily
linking them to practice.
• Descriptive studies – reviewing and evaluating existing theory and
knowledge in a field or describing particular situations or events. This
might include testing existing theories, describing the state of the art, or
looking for limits in previous generalisations.
• Exploratory studies – exploring a situation or a problem. These studies are
useful for finding out ‘what is happening; to seek new insights;
• Exploratory studies can be performed through literature searches, open
questionnaires and interviews. These studies can start out by exploring
particularly broad areas, concepts and ideas, before focussing in and
narrowing down to specifics as the research progresses.
 Classifying research
• Causal studies – assessing the effects that one or more variables have on
another. The independent variables are those which might be having an
influence on the dependent variable in which you are interested
 Research methods
• action research,
• experiment,
• case study and
• survey
 Interviews
• Selecting interviewees.
• Preparation. Don’t just expect to interview somebody ‘off the top of
your head’.
• Questionnaires.
• Have an agenda for the interview.
• Structured replies. In some cases you might wish to quantify an answer
from the interviewee.
• Note taking.
• Confidentiality.
• Logistics of the session.
• Characterisation.
• Ethical issues.
Research Methodology
 What is a methodology?
• How to collect and analyse data/information
about your topic (that is, how to carry out the
empirical part of a research project)
– What sort of data do I need, eg, quant or qual?
– Where is it located, eg, my firm or overseas?
– How do I collect it, eg, survey or a company case?
– What will I do with the data, that is, how will I analyse it?

39
Core issue: How many = quant? Why =
qual?
• Quant. Arthur wanted to know how relationships were
built in internet marketing rather than in the well-
researched field of non-internet banking. He surveyed 600
customers of a bank and analysed the data to get precise
measurements of some key variables.
• Qual. John was an insurance broker. He wanted to improve
his own firm's customer service and the literature could not
help him. He conducted interviews with selected managers,
staff and customers of 8 case study broking firms, and built
a theory to help him understand his problem.

40
Some common research methods

• quantitative approach
- surveys
• qualitative methods
- case studies (single/multiple)
- action research
• mix of quant and qual (but is
this recommended?)

41
 Quant vs qual
• numbers • words
• big sample • small sample
• theory testing • theory building

42
 Direct link between research problem and method,
data collection, analysis

Your research must show a clear link

between your research problem and
• research method to be used
(survey/case/action research)
• actual data to be collected (how,
when, where, what kind)
• analysis of data (how, what)
43
 So, to justify a research method

The researcher must use an ‘appropriate’ methodology

for his or her research problem
Argument:
• each method has inherent weaknesses and strengths
• any research problem has particular features
• thus choice of method depends on research problem

This is the most common way

of justifying a research method in academic research

44
 Identifying the appropriate method
for addressing your research problem

• study the methods used in studies

similar to yours
• discuss with supervisor
• learn about research methods

45
 Data collection

• it may be possible to use existing data

• evaluate sources of new data, for example,
is it accessible (contacts, databases, field samples)
• consider time available and costs
• consider your skills and time to learn new data
collection and analysis skills
• focus on relevant data (do I really need this?)

46
 Data analysis

• read about data analysis

• familiarise yourself with appropriate analysis methods
• identify the relevant tools/techniques
• formulate comprehensive strategy for analysis
of your data (prioritise, plan, link analysis with research
questions or issues/hypotheses or propositions developed in
the lit review)
• keep analysis relevant
• be thorough and methodical
47
In brief, while selecting your research method
for your research blueprint...

• Yourself
- what is my philosophical outlook?
- what are my existing skills?
- do I have the time/want to branch out and
learn something new?
• Research problem (questions/hypotheses/propositions)
- does my problem lend itself to this approach?
- can I get access to people/organisations/respondents?
- will the data collected in this way provide meaningful
findings?

48
 Recap: core issue and agenda
• Core issue: Should I use a quantitative or a qualitative
methodology? The answer depends on your research
problem.
• Agenda:
– what is a methodology?
– differences between quant and qual methods
– matching your research problem and the
methodology
– data collection and analysis
– research proposal
49
 Conclusions
• There are many ways to skin a cat. Your main
decision is to select the type of cat you want
to skin, that is, your research problem
• Match your research problem/topic with your
methodology
• Then check if you can collect and analyse the
data of that methodology

50
 Identifying a research topic

06/20/2010 Version 1.0 51

 Identifying your research topic

• identify academic research issue

- research suggestions in published articles
- past dissertations/dissertation abstracts
- discussions with academics
- participate in funded research

• identify practical problem

- your own experience/knowledge
- trade or professional organisations
- professional and executive seminars

06/20/2010 Version 1.0 52

 Finalising your research topic

What is a ‘good’ topic?

• of interest to you
• with an existing body of knowledge
- this helps when working on Literature chapter
- serves as foundation for your study
• a sustainable topic (i.e. avoid ‘hot topic’)
• builds on your existing knowledge/skills

06/20/2010 Version 1.0 53

 Refining the research topic

06/20/2010 Version 1.0 54

 From research area to research questions

research area

research problem

research questions
narrowing or increased
down hypotheses focus

06/20/2010 Version 1.0 55

 Research area, problem, question

Research area =
- general research area which you are interested in

Research problem =
- specific issue which you are investigating

Research questions/hypotheses =
- actual questions you try to answer
- specific hypotheses you plan to test

06/20/2010 Version 1.0 56

 From research area to research questions

Research area

Research problem

Research questions
or hypotheses

06/20/2010 Version 1.0 57

 Research area, problem, question
- an example

Research area =
electronic commerce

Research topic/problem =
design of effective e-commerce web sites

Research question/hypotheses =
how does web designer training affect performance
of web sites?

06/20/2010 Version 1.0 58

 Getting from general research area to
more specific research problem

• immerse yourself in the relevant literature

• discuss with supervisor
• talk with other research students
• carry out a preliminary study

06/20/2010 Version 1.0 59

Getting from research problem to research
questions or hypotheses

• identify sub-issues within the problem and

phrase a question or hypothesis for each

• how to identify sub-issues?

- use the literature
- discuss with supervisor

06/20/2010 Version 1.0 60

 Finalising the research questions

• minimum criteria for a research question

- do-able (i.e. consider research method)
- publishable (i.e. should contribute to knowledge)

• aim for symmetry of findings in question

(should be useful research irrespective of findings)
- for hypotheses: interesting - whether ‘yes’ or ‘no’ finding
- for questions: interesting - whether outcome is pos/neg;
high/low; much/little; etc

06/20/2010 Version 1.0 61

 Research topic/problem
and research question/hypotheses

- how do they link to the research thesis?

06/20/2010 Version 1.0 62

 Problem, questions and the thesis

The research problem -

• identifies the relevant literature (Literature
chapter)

The research questions/hypotheses -

• indicate appropriate method of investigation
(Method chapter)
• identify data to be collected, analysed and
discussed (Findings chapter)

06/20/2010 Version 1.0 63

 Thesis should be built
on solid foundations

Your thesis
research topic/problem and research question/hypo
(agreed with supervisor)

Findings from relevant Problem in practical or

published literature professional setting

06/20/2010 Version 1.0 64

 Research Methods

06/20/2010 Version 1.0 65

 When is case research appropriate?

• lack of current knowledge about the issue

- literature review shows current knowledge has gaps
- hence: need for theory building
- case research accepted as theory building research method

• complexity of phenomenon studied

- need to understand full context
- need for rich understanding of the issues

• phenomenon is contemporary and dynamic

06/20/2010 Version 1.0 66

 Skills required for case research

A case researcher should

• be able to ask good questions
• be a good ‘listener’
• be adaptive and flexible
• have a good grasp of the issues being studied
• be unbiased by preconceived notions

06/20/2010 Version 1.0 67

 Quality of the research

• discuss the issues

- define validity, reliability, generalisability
- explain what it means in the context of case research
- explain how each is addressed in your study

• use references
- for specific case quality issues: Yin, 1994, p33

06/20/2010 Version 1.0 68

 Quality of research - the four issues

• construct validity
using correct measures to study the research concepts
• internal validity
establishing causal relationships
• external validity
generalising to larger domain
• reliability
demonstrating that study can be repeated with same results

06/20/2010 Version 1.0 69

 Quality of the research -
methods of enhancing case study research quality

Construct validity - use multiple sources of evidence

- establish chain of evidence
- informants review draft of report
Internal validity - pattern matching
- explanation building
External validity - use replication logic
Reliability - use protocol
- develop and maintain data base
(adapted from Yin, 1994, p33)

06/20/2010 Version 1.0 70

 Ethical considerations

Identify the issues and explain how each is addressed:

• confidentiality of information

• privacy of informants

• etc

06/20/2010 Version 1.0 71

 What’s the Difference Between
“Method” and “Methodology”?
Method: Methodology:
• Techniques for • The underlying theory
gathering evidence and analysis of how
• The various ways of research does or should
proceeding in gathering proceed, often
information influenced by discipline

06/20/2010 Version 1.0 72

 EMPERICAL RESEARCH REPORTS

06/20/2010 Version 1.0 73

 WHAT IS AN EMPIRICAL RESEARCH REPORT?

1:An empirical research report is structured to answer specific

questions posed by readers in scientific and technical fields:
–What is the problem?
–What was done to study the problem?
–What was found?
–What do the findings mean ?
(Introduction to Technical Writing: Process & Practice, by Lois
Johnson Reid)
2:An empirical research report is a report in which you gather your
most important information from primary sources, such as the field or
laboratory, rather than published documents. The research focuses on
why certain things happen.
(Power Tools for Technical Communication,McMurray, Harcourt
College Publications 2002)

06/20/2010 Version 1.0 74

 COMPONENTS OF AN EMPIRICAL RESEARCH
REPORT

1 Title
2. Authors: Affiliations
3. Abstract: Synopsis of study
4. Introduction: Literature review, statement of goals, research
questions, and hypotheses
5. Methods and Materials: Participants, measures, equipment,
statistical techniques, etc.
6. Results: Summaries and analyses of the measures obtained
7. Discussion & Recommendations : Interpretations and implications of
the study
8. References

06/20/2010 Version 1.0 75

 TITLE

The title is a concise summary of the empirical research

report. The title should convey appropriate information
about the study or studies presented in the report.

06/20/2010 Version 1.0 76

 AUTHORS

People who make a major contribution to the study are

listed as authors. You might want to do a background
search on the authors in order to determine the expertise
the researchers have.

Questions that have to be answered here:

– Who did the research?
– Is there an established protocol for the order of the
authors’ names?

06/20/2010 Version 1.0 77

 ABSTRACT

A report of an empirical study also includes an abstract. The

abstract is a brief but comprehensive summary of the empirical
research report. It includes a concise statement of the goal of the
research, outlines the methods, and presents the essential results
and conclusions.

Questions to be answered here:

– Why did you do the study?
– How was the study done?
– What did you find?
– Why are these findings important?

06/20/2010 Version 1.0 78

 INTRODUCTION

The introduction sets the research in a context (it provides the "big
picture"), provides a review of related research, and develops the
hypotheses for the research.

The purpose of the introduction is to describe the problem, develop

the theoretical and empirical background for the research questions,
and elaborate a rationale for all parts of the study.
Cont’d …

06/20/2010 Version 1.0 79

 INTRODUCTION…. Cont’ d

In order to understand why the research was conducted, you

need to ask yourself the following questions:

– What are the research questions?

– Where did these research questions come from?
– Is the research important? Why or why not?

These questions set up the context and rationale for the study.

06/20/2010 Version 1.0 80

 LITERATURE REVIEW
Analyze published studies relevant to the issue
under study.
Synthesize to avoid simply listing studies and
findings.
Cite studies with author(s) and date.
Avoid plagiarism by constructing a
comprehensive outline.

State Your Objective: Purpose of the Study

State Your Research Questions
State Your Hypothesis: What you thought you'd
find?

06/20/2010 Version 1.0 81

 METHODS & MATERIALS
The methods section is a description of how the research was
conducted, including who the participants were, the design of
the study, what the participants did, and what measures were
used.
The questions that will help you evaluate the method are:
– Have you explained the samples used in the study?
– Are the samples appropriate for the study?
– What is the research design?
– Is the design appropriate for the research question(s)?
– What are the measures?
– Are the measures appropriate for addressing the research
question(s)?
– What ethical considerations are important to address?

06/20/2010 Version 1.0 82

 RESULTS

The results section contains the summaries and analyses of the

measures obtained in the study. This is where the "answers" to the
research questions are found.

The following questions will help you evaluate the results:

– What are the main results of the study?
– Can the results be used to answer the research question(s)?
– Can the results be generalized beyond the context of the study?

You need to understand what the results are before you can think
critically about them. This can be a tough task if you don't know how
to interpret the results. A good way to start to understand the results
is to study the figures and tables. Then read the text for the
researchers' interpretations.

06/20/2010 Version 1.0 83

 DISCUSSION
The discussion section contains the interpretations and implications of
the study. There may be more than one study in the report; in this
case, there are usually separate Method and Results sections for each
study followed by a general discussion that ties all the research
together.

The discussion section should start with a summary of the most

important results and then follow with a discussion of how the results
address the research questions.

Consider these questions as you write the discussion section:

– What conclusions do the researchers draw from their results?
– Are the conclusions important?
– Why or why not?
Cont’d …
06/20/2010 Version 1.0 84
 DISCUSSION …. Cont’d

Explain your results:

– Did your results support your hypothesis?

– Did your results relate to your objective?
– Did your results interpret in light of other published results on the
subject?
– Did your results suggest directions for further research?
– Did your results discuss the limitations of your study?

This section, or area of the report, is also the place to make

recommendations or state ideas for further research.

06/20/2010 Version 1.0 85

 REFERENCES

The references section cites all the literature

reported in the article. The reference citations are
used to support statements made in the article.

06/20/2010 Version 1.0 86

 SOME TIPS
When preparing the report remember to :
1. Determine that the objective is a project involving discussion of
causes, effects, or both.
2. Define the audience and purpose (&/or describe problem and
background).
3. Perform the research.
4. Plan and develop graphics and tables.
5. Identify causes.
6. Identify effects.
7. Identify the relationships between causes and effects.
8. Discuss causes and effects.

06/20/2010 Version 1.0 87

 References for this slide

Introduction to Technical Writing: Process & Practice

by Lois Johnson Reid, Bedford/St.Martin's 1993:

Power Tools for Technical Communication

McMurray, Harcourt College Publications 2002

Wire Research:
http://wire.rutgers.edu/research_assignments_empirical_link.html

Online Technical Writing

http://www.io.com/~hcexres/tcm1603/acchtml/otherep.html#primres
ch

06/20/2010 Version 1.0 88

How To Read a
Research Paper
 Research Papers
• Primary form in which research results are
disseminated in computer science
• Conference papers (shorter)
• Journal papers (longer)
– Often the complete version of a conference paper
– May come out several years after the conference
paper
 Reading a Paper Critically
• Understand the problem
• Understand the proposed solution
• Understand competing approaches / designs
• Evaluate the paper

• Peer review is the cornerstone of the scientific

publishing process
 Why?
• Learn to do research
• Learn to think critically about quality of research papers
– Someone will be thinking critically about your own work!
– In any discipline, there are fad and there are lasting ideas… learn to
tell the difference!
• Gain perspective
• Key issue: what are the questions to ask?
 Evaluating a Paper
• What is the problem being solved?
– Is it important? Relevant? Why?
– What is the prior work in this area?
• Is the proposed solution clever?
– Cleverness is orthogonal to importance!
• Are the assumptions and model reasonable?
• Impact
– Easier to evaluate for older papers
– Does other work build on it? Do other papers uses techniques and
solutions proposed in this paper?
 Evaluation Process
• Read slowly, take notes as you read
– Question assumptions, importance of the problem
– Write questions to track what you don’t understand
• Sometimes what is not in the paper is more important than what is in it
– Is there something the authors have overlooked?
• Don’t let ideas or design details pass until you understand them!
• Do not assume the paper is correct, even if published in a prestigious
peer-reviewed venue
 Ground Rules
• Try to understand
• Don’t be afraid to ask
• Be constructive
• Be polite
• Don’t be afraid to criticize (constructively!)
 types of research
primary secondary

scientist journalist
social scientist patent lawyer
historian R&D
Research Methodology
 Basic steps of a research
project
• Find a topicWhat, When
• Formulate questionsWhat, Why
• Define populationWho, When
• Select design & measurementHow
• Gather evidenceHow
• Interpret evidenceWhy
• Tell about what you did and found out
 Topic Ideas
• E-book usage
• Usability studies of
– Online tutorial(s)
– ‘My Library” portals
• Analysis of library web sites or library
instruction sites or pathfinders by best
practices
• Student learning outcomes in LI
programs
Types of methodologies
• QuaLitative Measures
– Descriptive
– Numbers not the primary focus
– Interpretive, ethnographic, naturalistic

• QuaNtitative Measures
– N for numbers
– Statistical
– Quantifiable
QuaNtitative measures

• CompareThings
• Count Things
• Survey People About Things
QuaNtitative measures
• Comparison studies
– Experimental and control groups
– Instructional methodologies (Colaric; Cudiner &
Harmon)
– Program assessment using before/after analysis
of research papers(Emmons & Martin)
 QuaNtitative measures
• Pre & Post Tests (Van Scoyoc)
• Measures & Scales
• Bostick’s Library Anxiety Scale (Onwuegbuzie & Jiao; Van Scoyoc)
• Procrastination Assessment Scale (Onwuegbuzie & Jiao)
 Keep In Mind That
• No study is perfect
• “All data is dirty is some way or another;
research is what you do with that dirty data”
(Manuel)
• Measurement involves making choices
 Getting Started

• Read to learn; read to analyze

– About research methodology
– Studies on similar topics
– Interesting studies
– Non-library studies
Core issue: How many = quant? Why =
qual?
• Quant. Arthur wanted to know how relationships were
built in internet marketing rather than in the well-
researched field of non-internet banking. He surveyed 600
customers of a bank and analysed the data to get precise
measurements of some key variables.
• Qual. John was an insurance broker. He wanted to improve
his own firm's customer service and the literature could not
help him. He conducted interviews with selected managers,
staff and customers of 8 case study broking firms, and built
a theory to help him understand his problem.

107
 We need both, though not necessarily in the
one research project
• Quant. If you were studying ‘who’ participated in
riots, and ‘how much’ damage had been done, you
might survey residents, examine business records (an
archival analysis), or conduct a ‘windshield survey’ of
the riot area.
• Qual. In contrast, if you wanted to know ‘why’ riots
occurred, you would have to conduct interviews and
draw upon a wider array of documentary
information. (Yin 1994)

108
 Core issue and agenda
• Core issue: Should I use a quantitative or a qualitative
methodology? The answer depends on your research
problem.
• Agenda:
– what is a methodology?
– differences between quant and qual methods
– matching your research problem and the
methodology
– data collection and analysis
– research blueprint
109
 Data collection

• it may be possible to use existing data

• evaluate sources of new data, for example,
is it accessible (contacts, databases, field samples)
• consider time available and costs
• consider your skills and time to learn new data
collection and analysis skills
• focus on relevant data (do I really need this?)

110
 Data analysis

• read about data analysis

• familiarise yourself with appropriate analysis methods
• identify the relevant tools/techniques
• formulate comprehensive strategy for analysis
of your data (prioritise, plan, link analysis with research
questions or issues/hypotheses or propositions developed in
the lit review)
• keep analysis relevant
• be thorough and methodical
111
In brief, while selecting your research method
for your research blueprint...

• Yourself
- what is my philosophical outlook?
- what are my existing skills?
- do I have the time/want to branch out and
learn something new?
• Research problem (questions/hypotheses/propositions)
- does my problem lend itself to this approach?
- can I get access to people/organisations/respondents?
- will the data collected in this way provide meaningful
findings?

112
 The research proposal
research problem/topic
with
questions or research blueprint
hypotheses/propositions = basis for Chap 1

• research problem/topic
research method • justification for topic
• evidence of lit. search
• res. questions/hypotheses
• proposed method and design
• timetable
• budget
113
 Recap: core issue and agenda
• Core issue: Should I use a quantitative or a qualitative
methodology? The answer depends on your research
problem.
• Agenda:
– what is a methodology?
– differences between quant and qual methods
– matching your research problem and the
methodology
– data collection and analysis
– research proposal
114
 Conclusions
• There are many ways to skin a cat. Your main
decision is to select the type of cat you want
to skin, that is, your research problem
• Match your research problem/topic with your
methodology
• Then check if you can collect and analyse the
data of that methodology

115
How To Read a
Research Paper
 Research Papers
• Primary form in which research results are
disseminated in computer science
• Conference papers (shorter)
• Journal papers (longer)
– Often the complete version of a conference paper
– May come out several years after the conference
paper
 Reading a Paper Critically
• Understand the problem
• Understand the proposed solution
• Understand competing approaches / designs
• Evaluate the paper

• Peer review is the cornerstone of the scientific

publishing process
 Why?
• Learn to do research
• Learn to think critically about quality of research papers
– Someone will be thinking critically about your own work!
– In any discipline, there are fad and there are lasting ideas… learn to
tell the difference!
• Gain perspective
• Key issue: what are the questions to ask?
 Evaluating a Paper
• What is the problem being solved?
– Is it important? Relevant? Why?
– What is the prior work in this area?
• Is the proposed solution clever?
– Cleverness is orthogonal to importance!
• Are the assumptions and model reasonable?
• Impact
– Easier to evaluate for older papers
– Does other work build on it? Do other papers uses techniques and
solutions proposed in this paper?
 Evaluation Process
• Read slowly, take notes as you read
– Question assumptions, importance of the problem
– Write questions to track what you don’t understand
• Sometimes what is not in the paper is more important than what is in it
– Is there something the authors have overlooked?
• Don’t let ideas or design details pass until you understand them!
• Do not assume the paper is correct, even if published in a prestigious
peer-reviewed venue
 Ground Rules
• Try to understand
• Don’t be afraid to ask
• Be constructive
• Be polite
• Don’t be afraid to criticize (constructively!)