What do questionnaire/interviews measure?

KNOWLEDGE & INFORMATION What someone knows?

VALUES & PREFERENCES

What someone likes?

ATTITUDES & BELIEFS

What someone thinks?

How to assess questionnaire/interviews validity?

TO WHAT EXTENT MIGHT A QUESTION: Influence respondents to show themselves in a good light? Influence respondents to attempt to anticipate what researchers what to hear or learn? Ask for information about respondents that they may not know about themselves?

How to ask the questions?

DIRECT QUESTIONS Ask someone whether or not he likes his job

INDIRECT QUESTIONS

Ask what he thinks of his job or selected aspects of it

Frank and open responses more Q

LIES HOW OBVIOUSLY THE QUESTIONS SOLICIT SPECIFIC INFORMATION

How to ask the questions?

SPECIFIC QUESTIONS Focus on object, idea as input to attitude belief, or concept
How he likes a particular teacher

NON-SPECIFIC

QUESTIONS

Questions that probe for more general areas

How he feels with a particular class taught by the teacher

SPECIFIC QUESTIONS MAY CAUSE RESPONDENTS TO BECOME CAUTIOUS TO GIVE LESS-HONEST ANSWERS

How to ask the questions?

PROVIDE FACTS Do not always elicit factual answers
What type of car he owns

REQUEST OPINIONS

Do not necessarily elicit honest answers

Preference for Toyota or Honda models

DUE TO FAULTY MEMORY OR A CONSCIOUS DESIRE TO CREATE A PARTICULAR IMPRESSION, FACTUAL OR OPINION QUESTIONS DO NO ELICIT HONEST ANSWERS SO MINIMIZE SOURCES OF BIAS

How to ask the questions?

QUESTION FORM Respondent provides an appropriate answer
Do you think that the OSEC should be lengthened? ___ Yes ___ No

STATEMENT FORM

Respondent indicates whether statement is true or false

The OSEC should be shortened. ____ Agree ____ Disagree

FORMATS ARE INDISTINGUISHABLE IN THEIR POTENTIAL FOR ELICITING HONEST RESPONSES SELECT FORMAT BASED ON RESPONSE MODE

How to ask the questions?

PREDETERMINED

QUESTIONS

Require respondents to complete list of questions

Age _____: Gender _____: Educational Attainment _____:

RESPONSE-KEYED

QUESTIONS

Subsequent questions may or may not call for answers

Are you a college graduate? ____ Yes ____ No (skip next Q)

POSSIBLE QUESTIONS KEYED TO THE RESPONSE WHAT? WHY? HOW?

How to answer the questions?

RESPONSE MODE

UNSTRUCTURED RESPONSES

Allows respondent to give a response in whatever form he chooses

What led you to go to OSEC? ________________________________________________________ Describe your feelings as you think of your partner. ________________________________________________________

EXERT LITTLE CONTROL OTHER THAN BY ASKING QUESTIONS AND LIMITING THE AMOUNT OF SPACE PROVIDED FOR THE ANSWER

How to answer the questions?

RESPONSE MODE

STRUCTURED RESPONSES

Offer answers and ask respondent to choose one

Place a check opposite each item in the following list to indicate your answer.

What are your chances of reaching your goal?

____ Excellent ____ Good _____ Fair _____ Poor _____ Very Poor

CENTERS ON THE TYPE OF RESPONSE THAT THE RESPONDENT IS ALLOWED TO MAKE

How to answer the questions?

RESPONSE MODE

FILL-IN RESPONSE

Transitional mode between structured & unstructured questions

What is your occupation? ___________________________________ In what school did you do your undergraduate work? ______________ Looking at the above picture, what word best describe the way it makes you feel? __________________________________________

ALTHOUGH REQUIRING RESPONDENT TO GENERATE RATHER THAN CHOOSE A RESPONSE, IT TYPICALLY LIMITS RANGE OF POSSIBLE RESPONSES BY LIMITING THE ANSWER TO A SINGLE WORD OR PHRASE

How to answer the questions?

RESPONSE MODE

TABULAR RESPONSE

Requires numbers, words, phrases or degree of agreement along a scale

Suppose you were offered an opportunity to make a substantial advance in a job. Indicate how important it would be in stopping you in making that advance. Would stop me Work harder Leave family
RESPONSE THAT INCLUDE A VARIETY OF INFORMATION AND ORGANIZES EITHER FILL-IN OR SCALED RESPONSES

Might stop me from making change

Wouldnt matter at all

How to answer the questions?

RESPONSE MODE

SCALED RESPONSE

Respondent express endorsement or rejection of an attitude statement

Degree of likelihood: ________________________________________________ Excellent Good Fair Poor Very Poor

Degree of frequency:

Always

Often

Seldom

Never

SCALED RESPONSES CAN YIELD INTERVAL DATA

How to answer the questions?

RESPONSE MODE

FREQUENCY QUESTIONNAIRE Instructions: All of the questions below are about what you actually do. If you Always do what the statement says, circle the 1 for A. If you Often do what the statement says, circle the 2 for O. If you Seldom do what the statement says, circle the 3 for S. If you Never do what the statement says, circle the 4 for N. There are no right or wrong answers for these questions. We are interested only in what you actually do. a. I think about what I will do when I finish OSEC.

1. A

2. O

3. S

4. N

b. I talk to officers to learn about their jobs. 1. A 2. O 3. S 4. N

How to answer the questions?

RESPONSE MODE

RANKING RESPONSE

Respondent to rank order series of statements based on a criterion

Rank the following activities in terms of their usefulness to you as you write your research objectives. (Assign the numbers 1 through 5, with 5 indicating the most useful activity and 1 indicating the least useful. If any activity gave you no help at all, indicate this by 0).

___ Initial discussion of research lecturer

___ Initial small-group activity ___ Weekly student sessions

___ Individual session with adviser

___ Examples of research objectives
RANKING FORCES RESPONDENT TO GIVE CRITICAL ESTIMATES OF THE VALUES OF ALTERNATIVES

How to answer the questions?

RESPONSE MODE

CHECKLIST RESPONSE

Respondent select one of the possible choices in a checklist item

I get most of my professional and intellectual stimulation from: Check one of the following blanks: ___ A. Lecturers ___ B. Academic Dean ___ C. Director
FORM OF RESPONSE DOES NOT REPRESENT A SCALE, RATHER THEY ARE NOMINAL CATEGORIES

How to answer the questions?

RESPONSE MODE

CATEGORICAL RESPONSE

Offers respondent only two possibilities for each item

Are you a veteran? ____ Yes ____ No

Research counseling does not begin early enough. ____ True ____ False

YES-NO DICHOTOMY IS OFTEN USED IN THE CATEGORICAL RESPONSE MODE. ATTITUDE-RELATED ITEMS MAY GIVE TRUE-FALSE ALTERNATIVES.

How to construct questionnaire/interviews?

WHAT QUESTIONS SHOULD YOU ASK?

WHAT FORMATS SHOULD YOU USE?

WHAT RESPONSE MODES SHOULD YOU EMPLOY?

SPECIFYING VARIABLES TO MEASURE

QUESTIONS REFLECT INFORMATION RELATED TO HYPOTHESES OR RESEARCH QUESTIONS

STUDY RELATING SOURCE OF OCCUPATIONAL TRAINING TO WORK PERFORMANCE: --------------------------------------------------------------------Occupational Training PNPA, PMA, University Work Performance Level High, Medium, Low

SPECIFY VARIABLES BY NAME

CHOOSING THE QUESTION FORMAT

DECISION IS WHETHER TO PRESENT ITEMS IN A WRITTEN QUESTIONNAIRE OR AN ORAL INTERVIEW

CONSIDERATION
Personnel

INTERVIEW
Interviewers Payments to interviewers Extensive Extensive Possible Great Limited Good Interviewer, instrument, coding, sample Quite limited Limited

QUESTIONNAIRE
Enumerators Postage/printing Limited Limited Difficult Limited to roster Extensive Poor Instrument, sample Fair Extensive

to collect data Major expense categories Opportunities for response keying Opportunities for asking Opportunities for probing Magnitude of data reduction Number of respondents reached Rate of return Sources of error
Overall

reliability Emphasis on writing skill

CHOOSING THE RESPONSE MODE

KIND OF INFORMATION DETERMINE THE MOST SUITABLE MODE RESPONSE MODE TYPE OF DATA ADVANTAGES DISADVANTAGES

Fill-in
Scaled Ranking Checklist or Categorical

Nominal
Interval Ordinal Nominal*

Limited bias Expanded flexibility

Easy to score Easy to score Forces discrimination Easy to score Easy to respond

Difficult to score
Time-consuming Potential for bias Difficult to complete Limited data and option

* May be interval when totaled CHOICE OF RESPONSE MODE SHOULD BE BASED ON THE MANNER IN WHICH THE DATA WILL BE TREATED

CHOOSING THE RESPONSE MODE

FILL-IN Group into ranges to provide nominal data for chi-square analysis Applicable to parametric statistical analysis since considered as interval data May provide less information since they generate ordinal data

SCALED

RANKING

RULE OF THUMB USE THE SCALED RESPONSE MODE SINCE THE INTERVAL DATA CAN ALWAYS BE TRANSFORMED INTO ORDINAL OR NOMINAL SCALE

PREPARING INTERVIEW ITEMS

1. Do you think that junior officers should be required to pass an examination in order to be promoted? ___Yes ___No 1.1 If no, why examination should not be given to officers? ______________________________________________ 2. Which of the following areas do you think is most important for the officers school to concentrate on?

___ a. Teaching police operational skills

___ b. Preparing officers for higher responsiblities ___ c. Teaching values and moral behavior

SPECIFY VARIABLES THAT YOU WANT TO MEASURE, CONSTRUCT QUESTIONS THAT FOCUS ON THESE VARIABLES

PREPARING QUESTIONNAIRE ITEMS

IS THIS WHAT I WANT TO MEASURE?

1. Check one of the following to show how much of the time you feel satisfied with your job. A ___ most of the time B ___ about half of the time

2. Describe your health since entering the police force. CIRCLE ONE.

Excellent

Good

Average

Poor

3. Did you receive training for the job which you are now employed. (CHECK ONE BELOW) ___ high school ___ college ___ armed forces

___ others (please specify) _______________________

PREPARING QUESTIONNAIRE ITEMS

IS THIS WHAT I WANT TO MEASURE?

Instructions: All questions are statements to which we seek your agreement. If you Strongly Agree with any statement, circle the 1. If you Agree but not strongly with any statement, circle the 2. If you Disagree but not strongly, circle the 3. If you Strongly Disagree with any statement, circle the 4. There are no right or wrong answers for these questions. We are interested only in how you feel about the statements. I believe it is important for me to participate in OSEC. 1. SA 2. S 3. D 4. SD

2. I believe OSEC can offer many achievements.

1. SA

2. S

3. D

4. SD

3. OSEC schoolwork bores me. 1. SA 2. S 3. D 4. SD

PREPARING QUESTIONNAIRE ITEMS

IS THIS WHAT I WANT TO MEASURE?

Name ________________________ Course Name ________________ Lecturer ___________________

SATIFACTION SCALE
Do you feel like skipping the class? ____1_________2___________3___________4___________5_____

Never

Rarely

Sometimes

Often

Always

2. Do you like this class? ____1_________2___________3___________4___________5_____ Very much Quite a bit Its all right Not so much Hate it

3. How much do you feel you have learned in this class? ____1_________2___________3___________4___________5_____ A great deal Quite a bit A fair amount Not much Nothing

INSTRUMENT DEVELOPMENT PROCESS

QUESTION HIERARCHY

MANAGEMENT QUESTION

Problem which management must answer

RESEARCH QUESTION

Answer to contribute solution to management problem

INVESTIGATIVE QUESTIONS

Specific questions to answer research question

MEASUREMENT QUESTIONS

Detailed questions which respondents must answer

INSTRUMENT DEVELOPMENT PROCESS

QUESTION CONSTRUCTION

CONTENT
SHOULD THE QUESTION BE ASKED? (significance to research topic) IS THE QUESTION OF PROPER SCOPE?

(coverage) CAN THE RESPONDENT ANSWER ADEQUATELY?

(subject of question) WILL RESPONDENT ANSWER ADEQUATELY? (question sensitivity)

INSTRUMENT DEVELOPMENT PROCESS

QUESTION CONSTRUCTION

WORDING
Is the question stated in terms of shared vocabulary? (common knowledge) Is the question clear?

(understandable) Is there biased wording?

(choice of word) Is there the right degree of personalization? (first or third person)