Deciding on the Research Questions

There is no foolproof, automatic way of generating research questions. While the sequence
envisaged here, of first deciding on a general research focus or area, and then refining that
down into a small number of relatively specific research questions, has an intuitive
reasonableness, things may not work out like this. A question, or questions, may come first
perhaps stimulated by theoretical concerns. You then seek an appropriate context, a
research focus, in which to ask the question. More commonly, as indicated above, the
general focus comes first.
There is evidence, however, that some ways of approaching the generation of research
questions are more likely to result in successful and productive enquiries than others. J. T.
Campbell et al. (1982) have looked at these issues by using a range of empirical techniques,
including contrast between studies judged by their originators as being either successful or
unsuccessful. Their remit was limited to research in industrial and organizational psychology,
but many of their conclusions seem to have general relevance to studies in the social
sciences.
An idea that emerges strongly from their work is that the selection of innovative research
questions is not a single actor r decision. Significant research is a process, an attitude, a way
of thinking. Significant research is accomplished by people who are motivated to do
significant research, who are willing to pay the cost in terms of time or effort (p. 109).
Box 3.4 list features considered by researchers to be associated with their successful and
unsuccessful projects. Campbell et al. View the choice process for selecting the research
questions as being often non-linear and involving considerable uncertainty and intuition.
Research starting with mechanistic linear thinking, closely to be of any significance.
However, something that starts out as poorly understood, given considerable theoretical
effort to convert if tint something which is clearly defined, logical and rational, could well be
of value.
Campbell et al. Also conducted a relatively informal interview study with investigators
responsible for what are considered important milestone studies in the study of
organizations, and reached conclusions which supported their previous ones. Specifically, it
did not appear that these milestone studies had arisen simply from seeking to test, or
extend, an existing theory previously used in that field of research. In fact, in virtually all
cases, the relevant theory or knowledge was imported from some other field. What was
clear was that these important studies were driven by some specific problem to be solved;
that they were characterized by a problem in search of a technique, rather than the reverse.
Each of the researchers was deeply involved in the substantive area of study, and it was
interesting to note that many of them reported Dan element of luck in either the creation or
the development of the research problem. However, it is well known in scientific creativity
that Lady Luck is more willing to bestow her favors on the keenly involved and well prepared
(see e.g. Medawar, 1979, p. 89)





Box 3.4
Features considered by researchers to characterize the antecedents of their successful and
unsuccessful research
Successful research develops from:
1. Activity and involvement Good and frequent contacts both out in the field and with
colleagues.
2. Convergence Coming together of two or more activities or interest (c. g. of an idea
and a method; interest of colleague with a problem or technique)
3. Intuition Feeling that the work is important, timely, right (rather than logical
analysis).
4. Theory Concern for theoretical understanding.
5. Real World valve Problem arising from the field and leading to tangible and useful
ideas.
Unsuccessful research starts with:
1. Expedience Undertaken because it is easy, cheap, quick or convenient.
2. Method or technique using it as a vehicle to carry out a specific method of
investigation or statistical technique.
3. Motivation by publication, money or funding Research done primarily for publication
purposes rather than interest in the issue.
4. Lack of theory without theory the research may be easier and quicker but the
outcome will often be of little value.


Developing the research questions
A know the area
It obviously helps to be really familiar with the area on which your research focuses. A good
strategy to force yourself into this position is to go public in some way produce a review
paper, do a seminar or other presentation with colleagues whose comments you respect (or
feral).
B Widen these base of your experience
You should not be limited by the research (and research questions) current in the specific
field you are researching. Researchers in other fields and from other disciplines may well be
wrestling with problems similar to yours, or from which useful parallels can be drawn. An
afternoons trawling through joumais in cognate disciplines one way. Contact and discussion
with practitioners may give a different perspective on what the question are.
C Consider using techniques for enhancing creativity
There is a substantial literature on creative and methods of promoting innovation which is
relevant to the process of generating research questions. Lumsdaine and Lumsdaine (1995)
provide comprehensive coverage. While their text is primarily addressed to engineers, there
is much of relevance here to real world researches. The methods include brainstorming (e. g.
Tudor, 1992; Rawlinson, 1981); the nominal group and Delphi techniques (e. g. Dulbecco,
1986); and focus groups. (See chapter 9 below, p.284).
Note: The techniques for enhancing creativity are primarily concerned with groups. Even if
you are going to carry out the Project on an individual basis, there is much to be sold for
regarding this initial stage of research as a group process and enlisting the help of others.
Consider, for example, the Delphi technique. In this contexts it might mean getting together
a group of persons, either those who are involved directly in the Project or a range or
colleagues with interest in the focus of the research. (Bear in mind the point made in section
B, that there is advantage in including in the group colleagues from other disciplines and
practitioners.). Each individual is then asked to generate independently, if. e. not in a group
situation, say, up to three specific research questions, perhaps giving a justification for the
questions chosen. The responses from each individual are collected, and all responses are
passed on in an unedited and unattributed from to all members of the group. A second cycle
then takes place. This might involve individuals commenting on other responses, and /or
revising their own contribution in the light of what others have produced. Third and fourth
cycles might take place, either of similar from or seeking resolution or consensus though
voting, to ranking, or categorizing responses. Endecott ET. Al. (1999) and Gibson (1998)
provide examples of different uses of the technique.
D Avoid the pitfalls of
Allowing a pre- decision on method or technique to decide the questions to be
asked. A variant of this conceding the use of computerized packages for statistical
analysis also worth flagging. Developing research questions simply on the basis that
they allow the use of particular package that you have amiable is almost as big a
research sin as designing and carrying out a fixed research design study that you
dont know how to analyses.
Posing research questions that cant be answered (either in general or by the
methods that is feasible for you to use).
Asking questions that have already been answered satisfactorily (deliberate
replication resulting from a condemn about the status of a finding is different from
ignorance of the literature).

E Cut it down to size
Thinking about the focus almost always leads to a set of research questions that is too large
and diffuse. Grouping questions together and constructing a hierarchy of sub-questions nest
within more general ones helps to bring some order. It is important not to arrive at
premature closure, even on a list of questions threatening to get out of hand. What
commonly happens is that something like a research programmer emerges, which has within
it several relatively separate research projects.
Howler, the time will come when you have to make hard decisions about where your
priorities are and in particular about what is feasible, given the time and other resources
that you have amiable. In fixed designs you need to have done this to a very large extent
even before you pilot; the role of the pilot is, among other things, to fine-tune the questions.
In flexible design, you keep things much more open when starting data collection. Even here,
though, it is wise to have a condemn for feasibility at an early stage. The flexibility comes in
modifying and developing the questions as data collection and analysis proceeds. An
important criterion for such development is getting a better understanding of what is likely
to be feasible as the research process continues.
Guidelines for the number of research questions you might be able address in a single study
vary from three to over ten. Obviously this depends on the nature of the specific research
questions and the resources at your disposal, but my experience is that very few small-scale
real world studies can cope adequately with more than six questions, and that four to six
such questions is a fair rule of thumb. Box 3.5 suggests characteristics of good research
questions.
F Think in terms of the purposes of you research
Clarifying the purpose or purposes of research can go a long way towards sorting out the
research questions. A tripartite classification is commonly used, distinguishing between
exploratory, descriptive and exploratory purposes. Following Marshall and Rossman (1999,
p33) it seems appropriate to and a fourth emancipatory category signaling the action
perspective present in many real world studies (see also the discussion in chapter 2 above,
p.28). These are summarized in box 3.6 a particular study may be concerned with more than
one purpose, possibly all four, but often one will predominate. The purpose may also change
as the study proceeds. Box 3.7 gives an example of links between research questions and
purpose.


BOX 3.5
Good research questions
Good questions are:
Clear they are unambiguous and easily understood.
Specific] they are sufficiently specific for it to be clear what constitutes an answer.
Answerable we can see heat data are needed to answer them and how those data
will be collected.
Interconnected the questions are related in some meaningful way, forming a
coherent whole.
Substantively relevant they are worthwhile, non-tribal questions worthy of the
research effort to be expended.


Box 3.6
Classification of the purposes of enquiry
1. Exploratory
To find out what is happening, particularly in Little-understood situations.
To seek new insights.
To ask questions.
To assess phenomena in a new light.
To generate ideas and hypotheses for future research.
Almost exclusively of flexible design.
2. Descriptive
To portray an accurate profile of persons, events or situations.
Requires extensive previous knowledge of situation etc. To be researched or
described, so that you know appropriate aspects on which to gather
information.
May be of flexible and/or fixed design.
3. Exploratory
Seeks an explanation of a situation or problem, traditionally but not
necessarily in the form of casual relationships.



Chapter four
Writing Strategies and Ethical Considerations
Before designing a proposal, it is important to have an idea of the general structure or outline
of the topics and their order. The structure will differ depending on whether you write a
quantitative, qualitative, or mixed methods project. Another general consideration is to be
aware of good writing practices that will help to ensure a consistent and highly readable
proposal (and research project). Throughout the project, it is important to engage in ethical
practices and to anticipate what ethical issues will likely arise. This chapter provides outlines
for the overall structure of proposals, writing practices that make them easy to read, and
ethical issues that need to be considered as the proposals are written.
WRITING THE PROPOSAL
Sections in a Proposal
It is helpful to consider the topics that will go into a proposal. All the topics need to be
interrelated and provide a cohesive picture of the entire project. An outline is helpful, but the
topics will differ depending on whether the proposal is for a qualitative, quantitative, or mixed
methods study. In this chapter, I advance outlines for sections of a proposal, as an overview of
the process. In individual chapters to follow, the sections will be detailed further.
Overall, however, there are central arguments that frame any proposal. They are introduced as
nine central arguments by Maxwell (2005). I pose them hare as questions to be addressed in a
scholarly proposal.
1. What do readers need, to better understand your topic?
2. What do readers know little about in terms of your topic?
3. What do you propose to study?
4. What is the setting and who are the people that you will study?
5. What methods do you plan to use to provide data?
6. How will you analyze the data?
7. How will you validate your findings?
8. What ethical issues will your study present?
9. What do preliminary results show about the practicability and value of the proposed
study?
These nine questions, if adequately addressed in one section for each question, constitute the
foundation of good research, and they could provide the overall structure for a proposal. The
inclusion of validating findings, ethical considerations (to be addressed shortly), the need for
preliminary results, and early evidence of practical significance focus a readers attention on
key elements often overlooked in discussions about proposed projects.
Format for a Qualitative proposal
In the light of these points, I propose two alternative models. Example 4.1 is drawn from a
constructivist/ interpretivist perspective, whereas Example 4.2 is based more on an advocacy/
participatory model of qualitative research.
Example 4.1 A Qualitative Constructivist / Interpretivist Format
Introduction
Statement of the problem (including existing literature about the problem, significance
of the study)
Propose of the study and how study will be delimited
The research questions.
Procedures
Philosophical assumptions of qualitative research.
Qualitative research strategy
Role of the researcher
Data collection procedures
Strategies for validating findings
Proposed narrative structure of the study
Anticipated ethical issues
Preliminary pilot findings (if available)
Expected outcomes
Appendixes: Interview questions, observational forms, timeline, and proposed budget.

In this example, the writer includes only two major sections, the introduction and the
producers. A review of the literature may be included, but it is optional, and, as discussed in
Chapter 3. The literature may be included to a greater extent at the end of the study or in the
expected outcomes section. I have added sections that may at first seem unusual. Developing
a timeline for the study and presenting a proposed budget provide useful information to
graduate committees, although these sections are typically not found in outlines for proposals.
Example 4.2 A Qualitative Advocacy/ Participatory Format
Introduction
Statement of the problem (including the advocacy/participatory issue being
addressed, existing literature about the problem, significance of study)
Purpose of the study and delimitations of the study
The research questions
Procedures
Philosophical assumptions of qualitative research
Qualitative research strategy
Role of the researcher
Data collection procedures (including the collaborative approaches used with
participants)
Data recording procedures
Data analysis procedures
Strategies for validating findings
Narrative structure
Anticipated ethical issues
Significance of the study
Preliminary pilot findings (if available)
Expected advocacy/ participatory changes
Appendixes: Interview questions, observational forms, timeline, and proposed budget
This format is similar to the constructivist/ interpretivist format except that the inquirer
identifies a specific advocacy/participatory issue being explored in the study (e. e.,
marginalization, empowerment), advances a collaborative form of data collection, and
mentions the anticipated changes that the research study will likely bring.
Format for a Quantitative Proposal
For a quantitative study, the format conforms to sections typically found in quantitative
studies reported in journal articles. The form generally follows the model of an introduction, a
literature review, methods, results, and discussion. In planning a quantitative study and
designing a dissertation proposal, consider the following format to sketch the overall plan (see
Example 4.3).
Example 4.3 A Quantitative Format
Introduction
Statement of the problem (issue, significance of issue)
Purpose of the study and delimitations
Theoretical perspective
Research questions or hypotheses
Review of the literature
Methods
Type or research design
Population, sample and participants
Data collection instruments, variables and materials
Data analysis procedures
Anticipated ethical issues in the study
Preliminary studies or pilot tests
Appendixes: Instruments, timeline and proposed budget
Example 4.3 is a standard format for a social science study, although the order of the sections,
especially in the introduction, may vary from study to study (see, for example, -miller, 1991;
Rudestam y Newton, 2007). It presents a useful model for designing the sections for a plan for
a dissertation or sketching the topics for a scholarly study.
Format for a Mixed Methods Proposal
In a mixed methods design format, the researcher brings together approaches that are
included in both the quantitative and qualitative formats (see Creswell y Plano Clark, 2007). An
example of such a format appears in Example 4.4 (adapted from Creswell y Plano Clark, 2007).
Example 4.4 A Mixed Methods Format
Introduction
The research problem
Past research on the problem
Deficiencies in past research and one deficiency related to the need to collect both
quantitative and qualitative data.
The audiences that will profit from the study
Purpose
The purpose or study aim of the project and reasons for a mixed methods study
The research questions and hypotheses (quantitative questions or hypotheses,
qualitative questions, mixed methods questions)
Philosophical foundations for using mixed methods research
Literature review (review quantitative, qualitative and mixed methods studies)
Methods
A definition of mixed methods research
The type of design used and its definition
Challenges in using this design and how they will be addressed
Examples of use of the type of design
Reference and inclusion of a visual diagram
Quantitative data collection and analysis
Qualitative data collection and analysis
Mixed methods data analysis procedures
Validity approaches in both quantitative and qualitative research
Researchers resources and skills
Potential ethical issues
Time line for completing the study
References and appendixes with instruments, protocols, visuals
This format shows that the researcher poses both a purpose statement and research questions
for quantitative and qualitative components, as well as mixed components. It is important to
specify early in the proposal the reasons for the mixed methods approach and to identify key
elements of the design, such as the type of mixed methods study, visual picture of the
procedures, and both the quantitative and qualitative data collection procedures and analysis.


Gorad, S. y C. Taylor 2004, Combining Methods in Educational and Social Research,
Maidenhead, England. Open University Press, pp. 42-57
Triangulation and combining methods across studies
In this chapther we describe two of what are probably the most common ways in which data
derived from different methods are combined triangulation between methods in one study,
and the combination of results across a number of studies using different methods, the
previous chapter provided a brief consideration of qualitative and quantitative approaches,
because this pattern follows tends to happen to combined methods research in practice
(Erzberger and Prein 1997). Research designs tend to split the two into exclusive parts (often
conducted by different individuals), and the try to relate the results to each other at the end.
In later chapters we consider more sophisticated and iterative designs.
The combination of evidence between studies using different methods is important because of
current demands to synthesize the evidence on practical topics within education and social
science, to engineer our findings as a community into a useable product for practitioners or
policy-makers. This is a difficult issue because there is no universally agreed way of doing it.
Most protocols for systematic reviews, for example, have tended to privilege numeric data
since this is purportedly easier to accumulate, average or meta-analyse across studies. The
results of each study can be converted into an effects size, and these can be averaged across
all studies, and weighted in terms of the size of each study. However, even where appropriate
protocols can be laid down for textual or visual data, these do not apply easily to numeric data
as well. Trying to find methods to integrate different types of data, or finding different
methods of analysis to use with the same data, also poses considerable problem of scale
(Erzberger and Prein 1997). In general, there is a tradeoff between the number of cases to be
studied ant the number and depth of the attributes that can be studied for each case. While
not providing in any way a total solution to this problem, this chapter develops the notion
Bayesian approaches to analysis introduced in Chapter 2, and illustrates how these can help
meld both qualitative and quantitative data into at least one kind of synthesis.
What is triangulation?
Triangulation between the evidence produced by different research methods is thought to be
a simple and common form of combined methods. Various reasons have been advanced for
the use of combined methods triangulation, including increasing the concurrent, convergent
and construct validity of research, the ability to enhance the trustworthiness of another, and in
testing hypotheses (Perlesz and Lindsay 2003). This section considers this metaphor/analogy of
triangulation in social science research. In particular we are concerned with triangulation in
mixed-methods approaches, such as those involving both traditionally quantitative and
qualitative techniques. Quantitative work, here, refers to counts and measures of things, but
not only work based on traditional statistical thcory or the standard frequentist approach.
Qualitative work predominantly uses words as data, but is not necessarily interpretative (see
chapter 2). As the types of data involved differ between the two approaches, so the
appropriate kinds of analysis often differ as well. We do not consider here the notion of
research paradigms as a barrier either to mixed methods work or to the triangulation of
results (but see Chapter 9).
The concept of triangulation between datasets is explained in introductory textbooks, and will
be familiar to most readers. The two or more different methods involved could both have
been generated within one study, which is triangulation at its simplest, or they could be
combined across two or more different studies (see below). However, the term triangulation
is also generally the source of considerable confusion (Kelle 2001). For example, we have
heard one respected source explain that triangulation involves the collection of data from
three vantage points, or the collection of three different kinds of data, in order to determine
something about a fourth phenomenon lying within the notional triangle formed by these
three points (or perhaps where the triangle is itself the metaphorical phenomenon to be
investigated by multiple methods). Most sources explain, rather, that triangulation involves
only a minimum of two vantage points or datasets to tell us something about a third
phenomenon. It is also typically explained using a metaphor derived from an analogous
trigonometric process during land surveying.
In land surveying, we could determine the position of a third point C if we know the positions
of points A and B, and we also know the interior angles (X and Y) at any two points of the
triangle thus formed by A, B and C (see Figure 3.1). in this case, C is the Phenomenon we are
trying to investigate (the unknown), while A and B are our two observations (or two different
methods when we are combining both qualitative and quantitative evidence).
Figure 3.1 two point provide triangulation.


Alternatively, we can imagine the metaphor in terms of perspectives (see Figure 3.2). C is still
the phenomenon to be investigated. A and B are our two perspectives (observation points or
methods). Each isolated view of C may produce a two-dimensional picture (such as a circle or a
rectangle), but when put together with binocular (triangulated) vision they produce a three-
dimensional image (such as a cylinder, with the circle as its face and the rectangle as its
side).

Figure 3.2 two perspectives provide triangulation
If either of these are close to the analogy for social science triangulation, then several
important conclusions can be drawn. First, the whole process assumes that is a relatively
stable, genuine observable phenomenon to be investigated. Therefore, the process explicitly
rules out both positivism (the belief that the existence of objects stems solely from their
measurement, Cook and Payne 2002) and relativism (the belief that objects do not have an
external reality, and that there can, therefore, be genuine multiple realities, Sale et al. 2002).
Second and again using the metaphor as it usually constructed, and against popular practice
(at least in writing about it, in surveying, and in the differing perspectives model of
triangulation, the whole process depends on all of the observations taken being accurate. If,
for example, we are trying to find a position for point C from the positions of, and interior
angles at, points A and B, then any error in our information about A or B will lead to an error
for C. similarly, then, in social science two different sets of observations (whether collected by
the some different methods) cannot be used both to check up on each other and for
triangulation. We can, of course, replicate our previous research, and even attempt near-
replication with different methods (what Erzberger and Prein 1997 refer to as convergence).
If the two components of the replication simply becomes another term for such replication.
But if the two components lead to substantially different results (such that we would conclude
that we have not produced replication), then we cannot use the two components to fix a third
because we do not know which of the two, if any, is in error. And even if we did know which
was in error, we would then end up with only one set of valid measurements for the position
of a point, and the analogy of trigonometry cannot help us.
Third, therefore, if triangulation means anything in social science terms it is about
complementarity, and nothing at all to do with mutual validation. The two observations or
methods must be directed at different aspects of the wider phenomenon to be investigated.
One of the methods might be indirect or reductionist in nature (a very valuable approach in
science, Verschuren 2001), and the other direct or holistic. In this case, we will obviously
expect that the results of the two observations will differ from each other (and will not be used
for mutual confirmation). If we are using two different methods then the results have to be
genuinely combined if something new is to result (as in the analogy of seeing a cylinder though
binocular vision). Perhaps, therefore, the explanation of perspectives is easier to follow. When
we view an object from two perspective, or study a social phenomenon using two methods,
then we expect to find something new as a result whether that is point C, the binocular
vision of a cylinder, a Gestalt, or simply a more well-rounded theory of the wider
phenomenon being investigated. This kind of combination reverts back to the true meaning of
the triangulation metaphor.
We cannot use two or more methods to check up on each other in a simple way. The methods
should be complementary, producing different aspects of the reality under investigation and
then put together. Or they could be designed to generate dissonance as a stimulus to progress.
As Perlesz and Lindsay (2003) report, little attention has been pain to resolving this problem of
dissonant data, perhaps because combined approaches are still so rare: we find it curious that
there are not more reports of data divergence in the literature (p.38).
For the present, consider Figure 3.3 If we assume that neither quantitative nor qualitative
approaches give us a complete picture of our object of study, that both will be valuable, and
that both can give us a differing partial picture, then the situation is a depicted. Some of the
object of study may be hidden to both perspectives. A qualitative approach gives us the
evidence available in sector A + C, whereas a quantitative approach gives us the evidence in
sector B+C. note that the intersect C may be empty but that, in so far as it exist, it represents
the traditional view of triangulation an overlap or confirmation between methods or
perspectives. Of course, C may represent the entire object of study (i.e. there would be
nothing more to find out about the given phenomenon). In this case all other findings using
these two perspectives are external to the object of the study, i.e. they are erroneous, noise,
or about a different phenomenon (and both A and B would be empty). The research may thus
generate findings unrelated to the object of study (the unshaded areas). These may be errors,
or they may be examples of those valuable serendipitous findings we all encounter when in
the field. However, Figure 3.3 also implies that each perspective provides further unique
evidence about the object of study (sectors A and B in isolation). We assume that segments A
and B are valuable parts of the object of study only available for inspection via one method.
Using the results in A, B and C as all valuable increases the amount and range of evidence
available to us. This is the power of combining methods.
Once familiar with this complementary notion of triangulation, we can use it to test our
exploratory theories. For example, it may be possible to draw predictions from our initial
theory not only about the likely contents of C, but also about what each approach will
generate separately (i.e. the contents of A and B). Our logic would be the standard one for
warranting conclusions (Gorard 2002c). if our theory is correct then we expect to find a set of
listed attributes in A, B and C. if we do not find these attributes in any sector, then this should
lead us to modify our theory.







Figure 3.3 A complementary combination of approaches.