The Layman's Guide to Social Research Methods

Introduction: Understanding the basics of social research methods can, at times, feel as if one is walking a very fine line between complete comprehension and abysmal failure. The terminology is complex, the concepts highly interrelated, and as we all know, every little detail matters when it comes to a successful thesis or dissertation. As such, I decided a sort of "cliff notes" to my experience in two good courses on this topic may be useful if not amusing! to others who are in the same position as I am " a weary eyed student about to begin a dissertation. It is, as you know a daunting task filled with both anticipation and a lot of fear. Contents:This web page provides an overview of the key elements of social research methods and is comprised of four different sections for your use. The first section is my attempt to explain in #nglish! the primary differences between reliability and validity. This section includes a discussion of threats to many types of validity " as all $ornell students in %ill Trochim&s check out his $enter for 'ocial (esearch )ethods! classes have been brainwashed to obsess about them but we know it is for our own good!. In addition, the second section provides a brief discussion on two fundamental ways to increase the strength of your analysis " statistical power, and research design. I&ve also compiled a glossary of some of the terminology that we must face and know! to appear credible as fledgling social scientists. *inally, I&ve tried to pool together some resources I have gathered over the last two years " both printed materials and additional web sites that may be helpful in understanding these concepts.

(eliability and +alidity, -hat&s the .ifference/

Reliability Definition: (eliability is the consistency of your measurement, or the degree to which an instrument measures the same way each time it is used under the same condition with the same sub0ects. In short, it is the repeatability of your measurement. A measure is considered reliable if a person&s score on the same test given twice is similar. It is important to remember that reliability is not measured, it is estimated. There are two ways that reliability is usually estimated, test1retest and internal consistency. Test/Retest Test1retest is the more conservative method to estimate reliability. 'imply put, the idea behind

test1retest is that you should get the same score on test 2 as you do on test 3. The three main components to this method are as follows, 2.! implement your measurement instrument at two separate times for each sub0ect4 3!. compute the correlation between the two separate measurements4 and 5! assume there is no change in the underlying condition or trait you are trying to measure! between test 2 and test 3. Internal Consistency Internal consistency estimates reliability by grouping 6uestions in a 6uestionnaire that measure the same concept. *or example, you could write two sets of three 6uestions that measure the same concept say class participation! and after collecting the responses, run a correlation between those two groups of three 6uestions to determine if your instrument is reliably measuring that concept. 7ne common way of computing correlation values among the 6uestions on your instruments is by using $ronbach&s Alpha. In short, $ronbach&s alpha splits all the 6uestions on your instrument every possible way and computes correlation values for them all we use a computer program for this part!. In the end, your computer output generates one number for $ronbach&s alpha " and 0ust like a correlation coefficient, the closer it is to one, the higher the reliability estimate of your instrument. $ronbach&s alpha is a less conservative estimate of reliability than test1retest. The primary difference between test1retest and internal consistency estimates of reliability is that test1retest involves two administrations of the measurement instrument, whereas the internal consistency method involves only one administration of that instrument.

Validity Definition:+alidity is the strength of our conclusions, inferences or propositions. )ore formally, $ook and $ampbell 2898! define it as the "best available approximation to the truth or falsity of a given inference, proposition or conclusion." In short, were we right/ :et&s look at a simple example. 'ay we are studying the effect of strict attendance policies on class participation. In our case, we saw that class participation did increase after the policy was established. #ach type of validity would highlight a different aspect of the relationship between our treatment strict attendance policy! and our observed outcome increased class participation!. Types of There are four types of validity commonly examined in social research. Validity:

2. $onclusion validity asks is there a relationship between the program and the observed outcome/ 7r, in our example, is there a connection between the attendance policy and the increased participation we saw/

3. Internal +alidity asks if there is a relationship between the program and the outcome we saw, is it a causal relationship/ *or example, did the attendance policy cause class participation to increase/ 5. $onstruct validity is the hardest to understand in my opinion. It asks if there is there a relationship between how I operationali;ed my concepts in this study to the actual causal relationship I&m trying to study1/ 7r in our example, did our treatment attendance policy! reflect the construct of attendance, and did our measured outcome " increased class participation " reflect the construct of participation/ 7verall, we are trying to generali;e our conceptuali;ed treatment and outcomes to broader constructs of the same concepts. <. #xternal validity refers to our ability to generali;e the results of our study to other settings. In our example, could we generali;e our results to other classrooms/ Threats To Internal Validity There are three main types of threats to internal validity " single group, multiple group and social interaction threats. Sin le Group Threats apply when you are studying a single group receiving a program or treatment. Thus, all of these threats can be greatly reduced by adding a control group that is comparable to your program group to your study. A History Threat occurs when an historical event affects your program group such that it causes the outcome you observe rather than your treatment being the cause!. In our earlier example, this would mean that the stricter attendance policy did not cause an increase in class participation, but rather, the expulsion of several students due to low participation from school impacted your program group such that they increased their participation as a result. A Maturation Threat to internal validity occurs when standard events over the course of time cause your outcome. *or example, if by chance, the students who participated in your study on class participation all "grew up" naturally and reali;ed that class participation increased their learning how likely is that/! " that could be the cause of your increased participation, not the stricter attendance policy. A Testing Threat to internal validity is simply when the act of taking a pre"test affects how that group does on the post"test. *or example, if in your study of class participation, you measured class participation prior to implementing your new attendance policy, and students became forewarned that there was about to be an emphasis on participation, they may increase it simply as a result of involvement in the pretest measure " and thus, your outcome could be a result of a testing threat " not your treatment. An Instrumentation Threat to internal validity could occur if the effect of increased participation could be due to the way in which that pretest was implemented.

A Mortality Threat to internal validity occurs when sub0ects drop out of your study, and this leads to an inflated measure of your effect. *or example, if as a result of a stricter attendance policy, most students drop out of a class, leaving only those more serious students in the class those who would participate at a high level naturally! " this could mean your effect is overestimated and suffering from a mortality threat. The last single group threat to internal validity is a Regression Threat. This is the most intimating of them all 0ust its name alone makes one panic!. .on&t panic. 'imply put, a regression threat means that there is a tendency for the sample those students you study for example! to score close to the average or mean! of a larger population from the pretest to the posttest. This is a common occurrence, and will happen between almost any two variables that you take two measures of. %ecause it is common, it is easily remedied through either the inclusion of a control group or through a carefully designed research plan this is discussed later!. *or a great discussion of regression threats, go to %ill Trochim&s $enter for 'ocial (esearch )ethods. In sum, these single group threats must be addressed in your research for it to remain credible. 7ne primary way to accomplish this is to include a control group comparable to your program group. This however, does not solve all our problems, as I&ll now highlight the multiple group threats to internal validity. Multiple Group Threats to internal validity involve the comparability of the two groups in your study, and whether or not any other factor other than your treatment causes the outcome. They also conveniently! mirror the single group threats to internal validity. A Selection-History threat occurs when an event occurring between the pre and post test affects the two groups differently. A Selection-Maturation threat occurs when there are different rates of growth between the two groups between the pre and post test. Selection-Testing threat is the result of the different effect from taking tests between the two groups. A Selection-Instrumentation threat occurs when the test implementation affects the groups differently between the pre and post test. A Selection-Mortality Threat occurs when there are different rates of dropout between the groups which leads to you detecting an effect that may not actually occur. *inally, a Selection-Regression threat occurs when the two groups regress towards the mean at different rates. 7kay, so know that you have dragged yourself through these extensive lists of threats to validity " you&re wondering how to make sense of it all. =ow do we minimi;e these threats without going insane in the process/ The best advice I&ve been given is to use two groups when possible, and if you do, make sure they are as comparable as is humanly possible. -hether you conduct a

randomi;ed experiment or a non"random study ""> ?7U( @(7UA' )U'T %# A' #BUI+A:#CT A' A7''I%:#D This is the best way to strengthen the internal validity of your research.The last type of threat to discuss involves the social pressures in the research context that can impact your results. These are known as social interaction threats to internal validity. Diffusion or "Imitation of Treatment occurs when the comparison group learns about the program group and imitates them, which will lead to an e6uali;ation of outcomes between the groups you will not see an effect as easily!. Compensatory Rivalry means that the comparison group develops a competitive attitude towards the program group, and this also makes it harder to detect an effect due to your treatment rather than the comparison groups reaction to the program group. Resentful Demoralization is a threat to internal validity that exaggerates the posttest differences between the two groups. This is because the comparison group upon learning of the program group! gets discouraged and no longer tries to achieve on their own. Compensatory E ualization of Treatment is the only threat that is a result of the actions of the research staff " it occurs when the staff begins to compensate the comparison group to be "fair" in their opinion, and this leads to an e6uali;ation between the groups and makes it harder to detect an effect due to your program. Threats to Construct Validity I know, I know " you&re thinking " no I 0ust can&t go on. :et&s take a deep breath and I&ll remind you what construct validity is, and then we&ll look at the threats to it one at a time. 7E/ 7E. $onstuct validity is the degree to which inferences we have made from our study can be generali;ed to the concepts underlying our program in the first place. *or example, if we are measuring self"esteem as an outcome, can our definition operationali;ation! of that term in our study be generali;ed to the rest of the world&s concept of self"esteem/ 7k, let&s address the threats to construct validity slowly " don&t be intimidated by their lengthy academic names " I&ll provide an #nglish translation. Ina!e uate "reoperational E#plication of Constructs simply means we did not define our concepts very well before we measured them or implemented our treatment. The solution/ .efine your concepts well before proceeding to the measurement phase of your study. Mono-operation $ias simply means we only used one version of our independent variable our program or treatment! in our study, and hence, limit the breadth of our study&s results. The solution/ Try to implement multiple versions of your program to increase your study&s utility. Mono-metho! $ias simply put, means that you only used one measure or observation of an important concept, which in the end, reduces the evidence that your measure is a valid one. The

solution/ Implement multiple measures of key concepts and do pilot studies to try to demonstrate that your measures are valid. Interaction of Testing an! Treatment occurs when the testing in combination with the treatment produces an effect. Thus you have inade6uately defined your "treatment," as testing becomes part of it due to its influence on the outcome. The solution/ :abel your treatment accurately. Interaction of Different Treatments means that it was a combination of our treatment and other things that brought about the effect. *or example, if you were studying the ability of Tylenol to reduce headaches and in actuality it was a combination of Tylenol and Advil or Tylenol and exercise that reduced headaches "" you would have an interaction of different treatments threatening your construct validity. Restricte! %eneraliza$ility &cross Constructs simply put, means that there were some unanticipated effects from your program, that may make it difficult to say your program was effective. Confoun!ing Constructs occurs when you are unable to detect an effect from your program because you may have mislabeled your constructs or because the level of your treatment wasn&t enough to cause an effect. As with internal validity, there are a few social threats to construct validity also. These include, 2. Hypothesis %uessing, when participants base their behavior on what they think your study is about " so your outcome is really not due solely to the program " but also to the participants& reaction to you and your study. 3.Evaluator &pprehension, -hen participant&s are fearful of your study to the point that it influences the treatment effect you detect. 5.E#perimenter E#pectancies, when researcher reactions shape the participant&s responses " so you mislabel the treatment effect you see as due to the program when it is more likely due to the researchers behavior. 'ee, that wasn&t so bad. -e broke things down and attacked them one at a time. ?ou may be wondering why I haven&t given you along list of threats to conclusion and external validity " the simple answer is it seems as if the more critical threats involve internal and construct validity. And, the means by which we improve conclusion and external validity will be highlighted in the section on 'trengthening ?our Analysis. Summary The real difference between reliability and validity is mostly a matter of definition. (eliability estimates the consistency of your measurement, or more simply the degree to which an instrument measures the same way each time it is used in under the same conditions with the same sub0ects. +alidity, on the other hand, involves the degree to which your are measuring what you are supposed to, more simply, the accuracy of your measurement. It is my belief that validity is more

important than reliability because if an instrument does not accurately measure what it is supposed to, there is no reason to use it even if it measures consistently reliably!.

'trengthening ?our Analysis

There are several ways to strengthen your analysis. The first, discussed in (eliability and +alidity, -hat&s the .ifference/, is to address as many threats to the validity of your research as possible. In addition, employing the methods discussed to achieve an accurate estimate of the reliability of your measure is also a good idea. This section will provide you with a brief overview of two other ways in which you can strengthen your analysis, basic statistics and research design. Statistics $onclusion +alidity relates to the statistical power of your study, and is the first type of validity to deal with in conducting any study " as it addresses the attempt to determine a relationship between your variables and your outcome. There are three primary ways to improve conclusion validity, 2.@ood reliability of your measure, as discussed in the first section of this web page4 3.@ood implementation usually achieved by standardi;ing the way in which your program is administered4 and 5.@ood statistical power. I&d like to focus a bit on statistical power. There are four primary components of statistical power, the sample si;e, the effect si;e, alpha level and power. Sample size is simply the number of people or units available to be studied. Effect Size is simply the ability to detect an effect relative to the other factors that appear in your study. &lpha level refers to the likelihood that what you observed is due to chance rather than your program. "o'er is the likelihood that you will detect an effect from your program when it actually happens. *or additional terms that are related to statistical inferences go to the @lossary of Terms. *inally, remember that the purpose of most research is to assess or explore relationships among a set of variables, and that the use of some straightforward, thought"out basic statistical methods can really enhance the strength of your findings.

Research !esi n The other way to strengthen your analysis is through a thought"out research design. There are three primary types of research designs, 2. Ran!omize! or true e#perimental designs, which are the strongest design for establishing a cause and effect relationship, as random assignment is used " and the groups involved are considered e6uivalent. This helps to reduce all of the multiple group threats to internal validity discussed earlier. 3.(uasi-E#perimental designs are those which employ multiple measures or a control group without randomly assigning participants to group. The ability of these designs to establish a cause effect relationship is dependent upon the degree to which the two groups in the study are e6uivalent. 5. )on-e#perimental designs do not employ multiple measure, do not use a control group and do not use any random assignment in its design. These are usually descriptive studies conducted using a simple survey instrument only once. -hile they are useful in their own right " they are weak in establishing cause1effect relationships. *or a comprehensive discussion of the variety of research designs that fit into all these categories, check out %ill Trochim&s $enter for 'ocial (esearch )ethods. *inally, a good research design is realistic in its scope relative to the resources of your study, appropriate for the context in which your treatment will occur, and involves a few good steps to measure your effect efficiently.

@lossary of Terms
Concurrent "alidity " the ability to distinguish between groups that should be theoretically distinguishable. Content "alidity " whether or not your instrument reflects the content you are trying to measure. Con"er ent "alidity " measures that should be related are related. .iscriminant validity " measures that should not be related are not. #ace Validity " addresses whether or not a measurement instrument is valid on its face. $redicti"e "alidity " the ability to predict something you want to predict.

%asic Statistical Terms Causal relationship " a relationship where variation in one variable causes variation in another. Correlation " a measure of the association between two variables, closer to 2 means a stronger correlation. Co"ariation " a measure of how two variables both vary relative to one another. !e"iation " the difference of a score from the mean. &rror Component " the part of the variance of an observed variable that is due to random measurement errors. 'ypothesis " a theory or prediction made about the relationship between two variables. Interaction " when the effect of one variable or factor! is not the same at each level of the other variable or factor!. Linear Correlation " a statistical measure of the strength of the relationship between variables e.g., treatment and outcome!. The closer the coefficient is to F2 or "2, the stronger the relationship " a positive correlation implies a direct relationship between the variables, a negative correlation implies an inverse relationship. Linear Re ression " the prediction e6uation which estimates the value of the outcome variable "y"! for any given treatment variable "x"!. Main &ffect " the effect of a factor on the dependent variable response! measured without regard to other factors in the analysis. Mean " the average of your sample, computed by taking the sum of the individual scores and dividing them by the total number of individuals sample si;e, "n"!. Median " if you rank the observations according to si;e, the median is the observation that divides the list into e6ual halves. Mode " the observation that occurs most fre6uently. (ull 'ypothesis " the prediction that there is no relationship between your treatment and your outcome. Random sample " a sample of a population where each member of the population has an e6ual chance of being in the sample. Si nificance le"el " the probability of finding a relationship between your treatment and effect when there isn&t one in reality.

Type I &rror " re0ecting the null hypothesis when it is true. Type II &rror " accepting the null hypothesis when it is false. Variation " a measure of the spread of the variable, usually used to describe the deviation from a central value e.g, the mean!. Cumerically, its the sum of the s6uared deviations from the mean.

Steps for a Successful Policy Analysis

)hat is $olicy *nalysis+

"The process through which we identify and evaluate alternative policies or programs that are intended to lessen or resolve social, economic, or physical problems." " $arl +. Aatton

$olicy *nalysis in Si, easy steps%ased on the ideas and approach followed by $arl +. Aatton there exists a very simple pattern of ideas and points to be considered in doing an actual policy analysis. The six steps are as follows, *+ ,erify- !efine- an! !etail the pro$lem+ The most relevant and important of them all because many times the ob0ectives are not clear or even contradictory from each other. A successful policy analysis will have allocated and identified clearly the problem to be resolved in the following steps. This is the foundation for an efficient and effective outcome of the whole process. The analyst must 6uestion both the interested parties involved as well as their agendas of the outcome. :ocating the problem in such a way that eliminates any ambiguity for future references. .+ Esta$lish evaluation criteria+ In order to compare, measure and select among alternatives, relevant evaluation criteria must be established. In this step it must be considered cost, net benefit, effectiveness, efficiency, e6uity, administrative ease, legality, and political acceptability. #conomic benefits must be considered in evaluating the policy. =ow the policy will harm or benefit a particular group or groups will depend on the number of option viable 7ptions more difficult than others must be considered but ultimately decided through analy;ing the parties involved with policy. Aolitical and other variables go hand in hand with the evaluation criteria to be followed. )ost of the time the client, or person or group, interested in the policy analysis will dictate the direction or evaluation criteria to follow. /+ I!entify alternative policies+ In order to reach this third step the other two must have been successfully reached and completed. As it can be seen, the policy analysis involves an incrementalist approach4 reaching one step in order to go on to the next. In this third step understanding what is sought is very important. In order to generate alternatives, it becomes important to have a clear understanding of the problem and how to go about it. Aossible

alternatives include the "do nothing approach" status 6uo!, and any other that can benefit the outcome. $ombining alternatives generates better solutions not thought of before. (elying on past experiences from other groups or policy analysis helps to create a more thorough analysis and understanding. It is important to avoid settling prematurely on a certain number of options in this step4 many options must be considered before settling into a reduced number of alternatives. %rainstorming, research, experiments, writing scenarios, or concept mapping greatly help in finding new alternatives that will help reach an "optimal" solution. 0+ Evaluate alternative policies+ Aackaging of alternatives into strategies is the next step in accomplishing a thorough policy analysis. It becomes necessary to evaluate how each possible alternative benefits the criteria previously established. Additional data needs to be collected in analy;ing the different levels of influence, the economical, political and social dimensions of the problem. These dimensions are analy;ed through 6uantitative and 6ualitative analysis, that is the benefits and costs per alternative. Aolitical 6uestions in attaining the goals are analy;ed as to see whether they satisfy the interested parties of the policy analysis. In doing this more concise analysis the problem may not exist as originally identified4 the actual problem statement from the first step may suffer a transformation, which is explained after evaluating the alternatives in greater detail. Cew aspects of the problem may be found to be transient and even different from the original problem statement. This modification process allows this method of policy analysis to allow for a "recycling" of information in all the steps. 'everal fast interactions through the policy analysis may well be more efficient and effective than a single detailed one. -hat this means is that the efficiency is greatly increased when several pro0ects are analy;ed and evaluated rather than 0ust one in great detail, allowing for a wider scope of possible solutions. Aatton further suggests to avoid the tool box approach, attacking options with a favorite analysis method4 its important to have a heterogeneous approach in analy;ing the different possible alternatives. It becomes inefficient to view each alternative under a single perspective4 its clearly relevant the need to evaluate each alternative following diverse evaluating approaches singled out according to the uni6ueness of each of them. 1+ Display an! !istinguish among alternative policies+ The results of the evaluation of possible alternatives list the degree to which criteria are met in each of them. Cumerical results don&t speak for themselves but are of great help in reaching a satisfying solution in the decision. $omparison schemes used to summari;e virtues are of great help in distinguishing among several options4 scenarios with 6uantitative methods, 6ualitative analysis, and complex political considerations can be melded into general alternatives containing many more from the original ones. In making the comparison and distinction of each alternative it is necessary to play out the economic, political, legal, and administrative ramification of each option. Aolitical analysis is a ma0or factor of decision of distinction among the choices4 display the positive effects and negative effects interested in implementing the policy. This political approach will ultimately analy;e how the number of participants will improve or diminish the implementation. It will also critici;e on how the internal cooperation of the interested units or parties will play an important role in the outcome of the policy analysis. )ixing two or more alternatives is a very common and practiced approach in attaining a very reasonably 0ustified policy analysis. 2+ Monitoring the implemente! policy+ Assure continuity, determine whether they are having impact. "#ven after a policy has been implemented, there may be some doubt whether the problem

was resolved appropriately and even whether the selected policy is being implemented properly. This concerns re6uire that policies and programs be maintained and monitored during implementation to assure that they do not change for unintentionally, to measure the impact that they are having, to determine whether they are having the impact intended, and to decide whether they should be continued, modified or terminated." )ainly, we are talking about internal validity4 whether our programs makes a difference, if there is no other alternate explanations. This step is very important because of the special characteristic that program evaluation and research design presents in this particular step. -illiam Trochim presents a very complete explanation of this concept. =is =ome Aage will be of great help in this matter

V*LI!IT.
%y Gretchen /- Rymarchy0

'ocial science research differs from research in fields such as physics and chemistry for many reasons. 7ne reason is that the things social science research are trying to measure are intangible, such as attitudes, behaviors, emotions, and personalities. -hereas in physics you can use a ruler to measure distance, and in chemistry you can use a graduated cylinder to measure volume, in social science research you cannot pour emotions into a graduated cylider or use a ruler to measure how big someone&s attitude is no puns intended!. As a result, social scientists have developed their own means of measuring such concepts as attitudes, behaviors, emotions, and personalities. 'ome of these techni6ues include surveys, interviews, assessments, ink blots, drawings, dream interpretations, and many more. A difficulty in using any method to measure a phenomenon of social science is that you never know for certain whether you are measuring what you want to measure. Validity is an element of social science research which addresses the issues of whether the researcher is actually measuring what s1he says s1he is. As an example, let us pretend we want to measure attitude. A psychologist by the name of Eurt @oldstein developed a way to measure "abstract attitude" by assessing several different abilities in brain in0ury patients, such as ability to separate their internal experience from the external world, ability to shift from one task to another, and the ability to recogni;e an oragni;ed whole, to break it into component parts, and then reorgani;e it as before. $arl Gung defined attitude a introversion and extraversion. (aymond $attell defined attitude in three components, intensity of interest, interest in an action, and interest in action toward an ob0ect =all H :ind;ey, 289I!.

Are any of these things what you think of when someone mentions the word "attitude/" .o any of these definitions of attitude seem like they are defining the same thing/ .o they seem "alid to you/ A definition of attitude that would seem to possess more validity to you might be the definition provided in the American Heritage Dictionary, "A state of mind or feeling with regard to some matter4 disposition" 28I9, p. 2<J!. This definition of attitude may appear to you to be the most "alid. +alidity in social science research has several different components " some people feel there are only three components of validity , and others feel there are four components of validity . 7n this page all four will be addressed. All of these facets of validity you would ideally want to establish for your research measure prior to administering it for your actual research pro0ect.

#ace "alidity re6uires that your measure appears relevent to your construct to an innocent bystander, or more specifically, to those you wish to measure. *ace validity can be established by your )om " 0ust ask her if she thinks your survey could ade6uately and completely assess someone&s attitude. If )om says yse, then you have face validity. =owever, you may want to take this one step further and ask individuals similar to those you wish to study if they feel the same way your )om does about your survey. The reason for asking these people is that people can sometimes become resentful and uncooperative if they think they are being misrepresented to others, or worse, if they think you are misrepresenting yourself to them. *or instance, if you tell people you are measuring their attitudes, but your survey asks them how much money they spend on alcohol, they may think you have lied to them about your study. 7r if your survey only asks how they feel about negative things i.e. if their car was stiolen, if they were beat up, etc.! they may think that you are going to find that these people all have negative attitudes, when that may not be true. 'o, it is important to establish face validity with your population of interest. In order to have a valid measure of a social construct, one should ne"er stop at achieving only face validity, as this is not sufficient. =owever one should ne"er skip establishing face validity, because if you do not have it, you cannot achieve the other components of validity. Content "alidity is very similar to face validity, except instead of asking your )om or your target members of your population of interest, you must ask experts in the field unless your )om is an expert on attitude!. The theory behind content validity, as opposed to face validity, is that experts are aware of nuances in the construct that may be rare or elusive of which the layperson may not be aware. *or example, if you submitted your attitude survey to Eurt @oldstein for a content validity check, he may say you need to have something to assess whether your respondents can break something down into component parts, then resynthesi;e it, as this is an important aspect of attitude, and otherwise you have

no content validity. *or an example of a study where a content validity check was used for an attitude assessment, click here . Another example measures influences , and another measures impacts . )any studies procede following content validity acvhievement, however this does not necessarily mean the measures used are entirely valid. Criterion "alidity is a more rigorous test than face or content validity. $riterion validity means your attitude assessment can predict or agree with constructs external to attitude. Two types of criterion validity exist, $redicti"e "alidity1 $an your attitude survey predict/ *or example, if someone scores high, indicating that they have a positive attitude, can high attitude scores also be predictive of 0ob promotion/ If you administer your attitude survey to someone and s1he rates high, indicating a posotive attitude, then alter that week s1he is fired from his1her 0ob and his1her spouse divorces him1her, you may not have predictive validity. Concurrent "alidity1 .oes your attitude survey give scores that agree with other things that go along with attitude/ *or example, if someone scores low, indicating that they ahve a negative attitude, are low attitude scores concurrent with happen at the same time as! negative remarks from that person/ =igh bolld pressure/ If you administer your attutude survey to someone who is cheerful and smiling a lot, but they rate low, indicating a negative attitude, your survey may not have concurrent validity. *or an extremely thorough example of research on the use of solution" focused group therapy with school children, which includes a concurrent validity check, click here .

*inally, the most rigorous validity test you can put your attitude survey through is the construct "alidity check. .o the scores your survey produce correlate with other related constructs in teh anticipated manner/ *or example, if your attitude survey has construct validity, lower attitude scores indicating negative attitude! should correlate negatively with life satisfaction survey scores, nd positively with life stress scores. These other constructs do not necessarily have to be predictive or concurrent, however often times they are. *or an in"depth discussion of construct validity, click here . To see what some of the threats are to construct validity, click here . If your attitude survey has made it this far, you have a most valid constructD Con ratulations

In search of truth throu h 2uantitati"e reasonin

Introduction 3uantitati"e reasonin becomes self1e"ident 4hen acti"ely participatin in the o"erall learnin e,perience- * research oriented learnin e,perience includes a formal and informal process of ainin 5 utili6in and systematically applyin 0no4led e to an area of interest in order to ma0e sense of the interrelationships bet4een 4hat one 0no4s and 4hat one learns)ith 2uantitati"e reasonin s0ills5 one can inte rate deducti"e lo ic aspects from multiple 0no4led e dimensions into pro ram e"aluation and research- There is a be innin 5 middle and an end to this cyclical process 4hich allo4s for the ad7ustment of additional information)hen approachin e"aluation 2uestions5 4ithin a particular conte,t5 it is important to 0eep in mind that a scientific5 linear model is but one method of or ani6in information- The purpose of this home pa e assi nment is to pro"ide a frame4or0 for constructin rele"ant 2uantitati"e interrelationships rounded on )illiam Trochim's $ro ram &"aluation and Research !esi n course offered throu h 'uman Ser"ice Studies 8'SS95 $lannin and &"aluation- Technolo ical ad"ances in computer use 4ill allo4 you5 the ne4 research student5 to follo4 and apply this information to your specific research pro7ect- This html assi nment 4ill introduce you to the 4orld of 2uantitati"e research methods5 as 4ell as open your eyes to the possibilities that e,ist for applyin similar concepts to 2ualitati"e studiesThere is tremendous "alue in understandin the plural dimensions of both 2uantitati"e and 2ualitati"e approaches to e"aluation methodolo ies- )olcott5 Guba5 and Lincoln5 ad"ocate the necessity of becomin familiar 4ith all other methods in order to appropriately select the method that best fits your area of research and desi n- The conte,t5 purpose5 and types of research 2uestions as0ed 4ill define the methodolo ical foundation of a study- /eepin this ca"eat in mind 4ill eliminate mismatched efforts and results that can only frustrate a be innin student in researchThe 2uality of one's research 4ill establish the foundation for the entire in2uiry process and is based on 2ualitati"e 7ud ments- :ud ments can be applied or transferred to 2uantitati"e terms 4ith both inducti"e and deducti"e reasonin abilities- ;3uantitati"e reasonin ---refers to a 4ide ran e of mental abilities; 8)olfe5 p- <9 that facilitates deducti"e reasonin in a "ariety of settin s- This article demonstrates the "alue of 2uantitati"e reasonin across many disciplines usin learned and intuiti"e s0ills that most indi"iduals in raduate pro rams already possess- 'o4e"er5 the burden is on the researcher to 7ustify the methodolo y and "alidity of their particular e"aluation or research area of interest- *s a student in both 2uantitati"e and 2ualitati"e courses in the field of 'SS5 I find it necessary to emphasi6e the importance of ainin a 4or0in 0no4led e of both methodolo ies 4ithin meanin ful conte,ts- Throu hout these collecti"e courses5 I ha"e been pro"ided opportunities to apply theoretical concepts to field assi nments that made more sense than simply readin about e,periences*fter considerin the conte,t and nature of your pro7ect5 then select the appropriate method of in2uiry to help direct the de"elopment of specific research 2uestions- The ob7ecti"e of your

in2uiry is to as0 2uestions in order to retrie"e the data or information that is salient to your pro7ect- Collectin and analy6in data 4ith 2uantitati"e strate ies includes understandin the relationships amon "ariables utili6in descripti"e and inferential statistics- This process 4ill re2uire a serious research student to ain a fuller 0no4led e base by underta0in courses in statistics or re ression analysis%riefly5 descripti"e statistics are theoretical postulates used to dra4 inferences about populationsand to estimate the parameters of those populations- Measures of central tendency and dispersion summari6e the information contained in a sample and are usually pro"ided in summary form5 such as distributions5 raphical and or numerical methods 8*pplied Re ression *nalysis for %usiness and &conomics5 =>>?9- Inferential statistics are based on descripti"e statistics and assumptions that enerali6e to the population from a selected sample- These assumptions focus on the use of continuous data and that the sample is a random representation of the population- Inferences made at lar e use probabilities and probability distributions- Statistical e"idence is especially important to policy ma0ers or other sta0eholders that ha"e a "ested interest in research/e"aluation pro7ects- $atton5 Guba @ Lincoln concur that sta0eholders use e,trapolated information as the basis of decision ma0in * ain5 as a reminder5 consider the conte,t of your o4n research and focus on the hypothesis enerated by your interests- *s0in empirical 2uestions in testable forms 4ill in"ol"e the traditional use of the (ull hypothesis "ersus the *lternati"e hypothesis- Test statistics for si nificance are used to determine if the null or alternati"e is to be accepted or re7ected- The null hypothesis tests for the differences bet4een population means- Inferential lo ic 4ill establish the standards of your study based on theory and application to realityTo effecti"ely e,press 2uantitati"e concepts re2uires familiarity 4ith the lan ua e- )olfe 8=>><5 p- <9 emphasi6es the importance of understandin and participatin in the entire process at all le"els of co niti"e reasonin - Readin 5 4ritin 5 and interactin 4ith the research process 4ill promote true learnin by inte ratin critical thin0in s0ills- )ith 2uantitati"e analysis5 it is especially important to understand the units of measurement in comprehensible formats5 such as "isual representations- Graphs5 charts5 plots5 and histo rams ade2uately display ra4 data for a i"en conte,t and chances of rememberin "isuals are reater than rememberin numbers or te,t- *c2uirin this 4or0in 0no4led e also includes s0ills in understandin scales and distributions- I hi hly recommend students enroll in the 'SS Measurement and !esi n course for an in1depth ;a4a0enin ; to the 4orld of nominal5 ordinal5 inter"al and ratio scales(o45 the bi 2uestion is ho4 does one arri"e at the ;appro,imation to the Truth based on conclusions from research+; 8Trochim5 'ome $a e5 /no4led e %aseA Validity9- Truth and in2uiry are a process related to lo ic5 e"idence5 and ar ument as Trochim 4ill elaborate on in his course presentations and /no4led e %ase- 'uman bein s interpret ra4 data and there are eneral uidelines presented to e"aluate assertions and to assess "alidity- 3uantitati"e strate ies use normal distributions based on statistical or re ression analysis- This appro,imation to normality 8truth9 is tan ible e"idence that assertions may be true'o4e"er5 if the constructs used to establish causality are not clearly operationali6ed to be in

4ith5 then those inferences about relationships and "ariables may not be "alid or reliable- It is lo ical to impro"e construct "alidity in order to stren then internal and conclusion "alidity* resource to address conclusion and internal "alidity issues for 2uantitati"e methods is Trochim's paper on Statistical $o4er and Statistical Tests 8=>BC9- This article describes ;four interrelated components that influence conclusions you mi ht reach from a statistical test in a research pro7ect ;8p- =9 utili6in a D , D decision matri,- The null and alternati"e hypotheses ;describe all possible outcomes 4ith respect to the inference; and is dependent upon the researcher determinin 4hich hypothesis ;allo4s the ma,imum le"el of po4er to detect an effect if one e,ists; 8Ibid- p-=9- Types I and II errors are firmly established in probability theory that one or the other hypothesis is the incorrect conclusion arri"ed at in the research- %asically5 a researcher 4ants to stat istically demonstrate that their pro ram did ha"e an effect in order to accept the alternati"e hypothesisSimultaneously5 understandin the theoretical bac0 round 4ithin the research construct can help eliminate systematic bias early on- $ost1$ositi"ism constructs support the lo ic of reasonin or decision1ma0in that parallel claims for e"idence of relationships- )hen constructin ar uments for "alidity5 it becomes increasin ly e"ident that one should attempt to control for or anticipate as many threats to "alidity as is humanly possible- )or0in 4ith an e,perienced research ad"isor as a mentor or facilitator in your research area 4ill be of tremendous "alue- See0 the ad"ise of those e,perienced in your field of interest and in"ite them to 7oin your team- There is a continuum of research perspecti"es a"ailable that can pro"ide "ariety5 clarity5 or "ision to your in2uiryEn this note5 desi n construction for 2uantitati"e or 2ualitati"e research pro7ects should be routinely considered throu hout the research cycle- Good desi n construction 8Trochim @ Land5 =>BD9 has se"eral characteristics that are applicable 4ithin eneral and specific conte,ts- &ffecti"e research strate ies should focus on indi"iduali6ed desi ns that are theory rounded5 are situational5 feasible5 redundant5 and efficient- *s part of the introduction to the 'SS ?>= course5 I 4ould recommend readin the article ; !esi nin desi ns for research; early in the semester to be in understandin 4here the abo"e issues fit in the o"erall scheme of research-

Summary of 2uantitati"e reasonin issues

To summari6e the material presented5 I ha"e offered a brief outline of interrelated concepts supportin 2uantitati"e reasonin 4ithin the methodolo y- There is "alue in learnin strate ies to systematically stren then the o"erall desi n and conclusions of research pro7ects- This approach has some transferable attributes that may also address 2ualitati"e studies throu h mi,ed methods- I can only spea0 from my personal e,perience that research5 in eneral5 is a "ery ri orous e,ercise in critical thin0in - !e"elopin plausible ar uments for inferences5 data collection and analysis strate ies5 and actually 4ritin or presentin

research findin s is a ma7or effort in any field that 4ill re2uire a commitment and compassion for your particular area of interest- The e"ol"in study one chooses to en a e in can pro"ide inestimable results to countless5 unseen others-

There are a fe4 issues re ardin this outline that ha"e not been addressed that merit some attention- Eften I am reminded that ;social research occurs in social conte,ts; and that not all human bein s enter social research 4ith the same le"el of 2uantitati"e or 2ualitati"e 0no4led e- /no4in personal stren ths and limitations can prepare a student in research methods for the amount of 4or0 re2uired to ain competency in 2uantitati"e research- *s a student masters elementary concepts5 an e,pansion of this 0no4led e can be applied to broader discussions in assertions and the de"elopment of e"idence to support those assertions- Ene must learn to thin0 deducti"ely and inducti"ely in order to "ie4 the similarities and differences of both methods 4hich can enhance your methodolo ical approach to research 2uestions- *bstract concepts from both methods are not mutually e,clusi"e5 as I ha"e obser"ed in the course of my study-

$resent research utili6in 2ualitati"e methods

*s mentioned earlier5 I am presently researchin en"ironmental efforts amon indi enous populations in the Fnited States across three eco eo raphical re ions- * brief conte,tual description of the Cornell *merican Indian $ro ram 8*I$9 research deser"es attentionEri inally5 the Commission for &n"ironmental Cooperation 8C&C9 solicited the participation of *I$ in assessin the e,tent of en"ironmental initiati"es 4ithin (ati"e *merican communities across the country- This entailed definin the constructs of ;indi enous roups5; ;sustainable efforts5; and ;cultural or indi enous 0no4led e systems-; The research desi n included a purposi"e sur"ey 4ithin a summati"e e"aluation research planT4enty 8DG9 descripti"e case studies 4ere selected that best represented nine eco eo raphical re ions of the country-

!ata trian ulation methods 4ere inte rated throu hout the research process in"ol"in a re"ie4 of rele"ant literature and e,istin pro ram documents5 as 4ell as utili6in technolo ical resources- * preliminary draft of ten of the t4enty case studies re"ealed se"eral inferences re ardin moti"ations for initiatin and maintainin en"ironmental efforts 4ithin indi enous communities- The predominant theme 4as that indi enous 0no4led e systems 4ere the impetus for community14ide responses to en"ironmental concerns-

)ithin this conte,t5 I am be innin to piece to ether the elements of research and desi n strate ies I ha"e ac2uired durin the course of my raduate pro ram- Grounded theory supports those "alidity issues I had the most concern 4ith- The internal debate I had about 2ualitati"e studies included 2uestions re ardin the "alidity and reliability of my pro7ect and the "alue of such a study in a lar er conte,t- I reali6ed that I 4as see0in e"idence to support a relationship bet4een cate ory and en"ironmental initiati"es-

*n indi enous 0no4led e system 8I/95 as a ma7or cate orical concept5 4as identified throu h each inter"ie4ee- This concept encompassed those "alues that had historical or cultural si nificance in the continuation of tradition 4hich included the de"elopment of an en"ironmental initiati"e that addressed the needs of that population 4ithin a specific re ion of the country- Specifically5 for the northeast re ion5 4here the Iro2uoia nations reside5 corn as a traditional food5 4as "ie4ed as ;an e,tremely unifyin concept and 4ay of life---4hich defined 8nati"e9 culture--- ;8Inter"ie4 >9- This inte rati"e cate ory e,tended into family5 ceremonial5 and community le"els 4hich facilitated the continuation of established "alues-

$atton 8=>>G95 Miles @ 'uberman 8=>>C95 Strauss @ Corbin 8=>>G95 and Guba 8=>B>9 pro"ided ;trust4orthiness; information that "alidated my research findin s- :ust as data spea0s for itself and emer es into themes and patterns5 so did my understandin that ;trust4orthiness; emer es throu h the efforts of the researcher to pro"ide credible5 confirmable and dependable findin s- Fsin an audit trail offered "isible support that the 8I/9 cate ory 4as inte rated into the o"erall research findin s of my pro7ect- Technical literature5 comparable inter"ie4s5 and other e,istin lin0a es in the en"ironmental net4or0 confirmed that traditional "alues 4ithin specific communities supplied the basis for pursuin sustainable efforts- These multiple sources addressed the internal and e,ternal "alidity issues of 2ualitati"e analysis that 4ere my initial concerns-

)olcott's 8=>>G9 article ;En see0in 1 and re7ectin 1 "alidity in 2ualitati"e research; proposed that 4hen one li"es throu h an e,perience5 the e,perience is "alidated- The synthesis of personal5 professional5 and spiritual elements of an e,perience pro"ides an ima e of 4hat is Real 8capital ;R;9- These elements are not compartmentali6ed in order to ha"e a fuller understandin of the comple,ity of relationships 4ithin systems- So it is 4ith the traditional "alues of nati"e peoples-

Conclusion

In conclusion5 I foresee mi,in methodolo ical approaches in establishin relationships bet4een the abo"e described "ariables- To statistically pro"ide e"idence of a relationship 4ould re2uire e,aminin separate nati"e populations5 based on comparable characteristicsThe feasibility of increasin the number of (ati"e communities in"ol"ed in this study may limit my scope5 ho4e"er I also reco ni6e the importance of determinin if there are other factors influencin a community's en"ironmental efforts- 'ey5 anythin is possible 4hen you ha"e a commitment to searchin for an appro,imation to the truthH

Ethical Dilemmas in Program Evaluation and Research Design

&li6abeth /- La$olt
As you browse through any college course description catalog you will inevitably come across courses in ethics. -hile many of the these classes are offered through philosophy departments, you will sometimes see them listed under biology departments or political science departments4 however, ethical issues should not be of concern only to future philosophers, doctors and politicians. #valuators of social programs must be prepared to face moral and ethical dilemmas at all stages of their work. Trochim defines evaluation as "the systematic ac6uisition and assessment of information to provide useful feedback about some ob0ect" 2882, 38!. At first glance this statement does not seem all that ethically daunting. -hat makes evaluation wrought with moral and ethical complications is the fact that it is people who design and perform the "systematic ac6uisition and assessment of information to provide" other people "useful feedback" about programs which are meant to, in some way, affect yet another group of people. -hen you give these people the titles of evaluators, evaluation audiences which could be funders, administrators or even politicians!, and program stakeholders, the potential for ethical complications to arise becomes evident. As an evaluator approaches a pro0ect, that person wants to ensure that the 6uality of their work is deemed acceptable by other social scientists and the evaluation community. The "theory of validity" is an approach to evaluation by which many evaluators set their standards. *our types of validity cumulatively contribute to this theory, conclusion validity4 internal validity4 construct validity4 and external validity. #thical 6uestions may arise as the researcher tackles each of these dimensions of validity.

$onclusion validity addresses the 6uestion of whether or not a relationship exists between two items. In terms of evaluation, this would be an analysis of whether or not a relationship exists between your program and the results you observed. -here are the ethical concerns here/ -hat could be wrong in trying to determine whether or not a nutrition education program taught people

to eat healthier or whether a support group helped women cope with breast cancer/ #thical issues first arise not in determining whether there is a relationship, but whether you want to be a contributor to researching that relationship. 7E. ?ou still don&t see the big deal. -ell, what if someone were to approach you and ask you to evaluate the effectiveness of a program which convinces minority women to have hysterectomies4 perhaps someone approaches you about doing an evaluation of how much torture a prisoner can withstand, would you take the 0ob/ )aybe theses scenarios seem a little far fetched to you, but if you examine your 3Jth $entury history books, I am sure you will find evidence of people calling for this type of research.! If those situations still seem a little ridiculous to you, think about some of the controversial social programs that are in place today. -ould you participate in the evaluation of a program that distributes clean hypodermic needles to drug addicts in the hopes of preventing the spread of =I+/ -ould you evaluate the success of a program which actively hands out condoms to teenagers/ Aerhaps your values are not in alignment with the point the evaluation wants to prove. %efore undertaking an evaluation you need to consider the ethical and moral implications of the research you are about to conduct. Are these evaluations that you would want to conduct/ -ould you do it and then say it was wrong or would you choose not to even be associated with the research/ Aerhaps you could establish construct validity, that there is a relationship, but what will the implication of your findings be/ -ould you want to contribute to the establishment of construct validity of an issue that does not meet your personal ethics and you feel does not contribute to the greater good of society/ These are 6uestions that you, as a social researcher, should stop to ponder.

In the American #valuation Association&s @uiding Arinciples for #valuators, principle III.# states, "#valuators articulate and take into account the diversity of interests and values that may be related to the general and public welfare" 28!. #rnest =ouse adds that, "evaluators should serve the interests not only of the sponsor, but of the larger society, and of various groups within society, particularly those most affected by the program under review" 53!. =e continues, "recogni;ing that there are interests to be served ties evaluation to the larger society and to issues of social 0ustice" 53!.

-ell, once you&ve determined that you can undertake an evaluation your mission is to actually prove that there is a relationship between your program and its outcomes, thereby establishing conclusion validity. There are three ways in which you can improve the likelihood of conclusion validity, ensure reliability, properly implement all testing procedures4 and establish good statistical power. -hile ethical concerns could present themselves when you are addressing reliability and instrumentation, it is with the concept of statistical power that I would like to address the possibility of ethical concerns presenting themselves.

There are four components of statistical power, sample si;e, effect si;e, alpha level, and power. Aower is exactly what we are looking for in most cases, we want to increase the odds of saying that there is a relationship between our program and the outcomes, when in fact, there is one. Unfortunately , by setting yourself up for high power, you are also increasing the odds of saying there is a relationship, when in fact, there is no relationship. This is called a type I error and is referred to as alpha. Alpha is a value which can be set by you, the evaluator and in this situation, the statistician!. ?ou can consider the value to be reflective of the level of risk you are willing to take in being wrong. This is where it is up to you to make a decision, and yes, it may turn into a situation where you will have to reflect upon your ethics. .etermining which is worse, a type I error or a type II error is forcing you to make a moral 0udgment, answering the 6uestion of what is right and what is wrong. Aerhaps it will help you to think about this in terms of the American 0ustice system. -ould you consider it worse to let a guilty person go free type II error! or is it our duty to keep innocent people from being punished type I error!. I highly recommend that you visit the 7G pages on this web site to follow up on this concept.! =ow you answer these 6uestions will depend on the nature of your evaluation, and what the implications are for your conclusions. 'o, I guess you&d like an example/ 'uppose you are evaluating a multi"million dollar government funded program which is supposed to help children from limited resource families do better in school. -hich scenario would be worse, canceling the program and saving taxpayers million of dollars by mistakenly determining that the program was ineffective or concluding that the program does work and help children, even through the program does not actually succeed in assisting them. -ho do you want to put at greater "risk", the taxpayers, whose tax dollars could be being wasted, or the children, who could lose out on a valuable program/ ?our alpha levels will reflect this, and it is your call as an evaluator to determine those levels. 7E, once you&ve established conclusion validity and grappled with the issue of power, it is time to move on and deal with internal validity, proving that it&s actually your program that&s making a difference. There are many threats to internal validity, all of which need to be addressed. 7ne of the single group threats is history. The simple fact that life brings new things into people&s lives daily can affect the internal validity of your study. Aeople do not stop existing outside of the constructs of your study. =ow many limitations can you put on a person you are studying in order to control for the possibilities of history threats/ -ould you tell a child that he or she can&t watch 'esame 'treet over the course of your study because you want to prove that it was your program, not %ig %ird or @rover contributing to that child&s growth/ 7f particular relevance to ethical issues are multiple group social interaction threats. ?ou&re trying to prove that it&s your program that&s making a difference, so you establish control or comparison groups. This has the potential to bring about compensatory e6uali;ation of treatment. Aerhaps the teacher of a class which has been chosen as comparison group sees what&s going on in the program classroom and she decides to do something extra for her class because she feels her class is missing out on something. =ow can you argue, on moral grounds, that she should deny her class that growth for the sake of research/ Aerhaps resentful demorali;ation occurs and the control group does worse, because they&re upset about not getting the program. -hat can you do to prevent this from happening/ ?ou could keep

the program a secret from the control group. Ah, but is that ethical/ Is it correct to not tell a group of people about the benefits of a program because you need to use them as a control, this is a dilemma particularly if there is significant evidence that the program you are evaluating is beneficial. Is it correct to deny the comparison group a treatment for the sake of research validity/ (ossi "findKsL it hard to envisage the circumstances under which doing so would not endanger the integrity of an evaluation. @iving out such information to a comparison or control group is the e6uivalent of shooting oneself in the foot, potentially narrowing the differences between them and treatment groups, correspondingly lowering the power of the evaluation" M9!.-hat kind of tradeoffs are you willing to make for the sake of social science research/ 7E, so you&ve muddled your way through some of the ethical dimensions of internal validity. Cow you have to face construct validity and determine if it is your program, all dimensions of your program and nothing but your program which is influencing the stakeholders. There are many threats to construct validity, one of which is "(estricted @enerali;ability Across $onstructs." In other words, what do you do when your program does work, but it it causing side effects, unanticipated conse6uences/ =ow will you address this dilemma/ I guess you could use another example. 7E. 'uppose you&ve devised a program which is intended to enable children to resolve their conflicts without violence. Aerhaps you&ve determined that for most of the children the program works, they talk out their problems more often, rather than pick fights. =owever, a certain portion of the population of children seem to be worse off from the program. They react even more violently then they had previously, they learn to like starting trouble with others and resorting to fist fights. .oes your program work, or doesn&t it, can you wholly establish construct validity/ In addition, there are several other threats to construct validity, some of which are social threats. These social threats include evaluation apprehension, hypothesis guessing and experimenter expectancies. These three threats all relate to whether or not you have told the study participants what your are studying. =owever, by not telling the study participants what you are studying violates the ethical and legal dimensions of voluntary participation and informed consent. The people you are studying should not be forced into participating. Additionally, the people you are studying must give consent to participate, fully understanding any risks that your study puts them under. %ut, what do you do if this affects the validity of your study/ 7E, I can tell you&re waiting for another example. -ell, perhaps you&ve heard about this study. I swear this was an actual study, but I am recalling it from memory. I do not have a reference for it , but I couldn&t resist using as an example here.! A study was done on how physical proximity affects people&s level of comfort. In other words, how much personal space do you need between you and another person without feeling uncomfortable/ -ell, the investigators studied the concept by hiding in the stalls of mens& bathrooms and recording the amount of time it took for men to urinate, depending upon how near or far another man stood at the urinals. The amount time was used as the indicator for level of comfortability. -ell, if you ask me, this is not only an invasion of privacy, but by no means did the researchers get voluntary participation or informed consent. -hat can I say, all for the sake of research/ 7nce you have established construct validity, it&s time to deal with external validity. =ow generali;able are your study findings/ $an the conclusions of your study be generali;ed to a larger

population of people/ Are the results representative of only the people who participated in the evaluation or is the information you collected applicable to a larger part of society/ "*ormally speaking the most representative samples will be those that are randomly chosen from the population, and it is possible for these randomly selected units to be randomly assigned to various experimental groups" $ook and $ampbell, 9M!. %ut, this is not necessarily feasible for all studies, this method, "can be followed for some issues where it is important to generali;e to particular target populations of persons, it is less clear whether it is often feasible to generali;e to target settings, except where these are highly restricted" 9M!. Aerhaps the one of the best ways to address the issue of ethical dilemmas in relation to external validity is by commenting on the debate between (egression .iscontinuity (.! .esign and (andomi;ed $linical trials ($T!. (. is a research design in which people are placed into a group based on a cutoff score. *or example, if you&ve developed a math tutoring program, you would place all of the students into the program who have scored below a specified score on some type of math ability test. This design, "intendKsL to balance ethical and scientific concerns when it is deemed unethical or infeasible to randomi;e all patients into study treatments" Trochim and $appelleri 5I9!. This design enables researchers to get their program to those who need its services the most. Trochim suggests that it forces politicians to use "accountable methods for assigning program recipients on the basis of need or merit" 288J, 23N!. =owever, there are drawbacks to the design. "The lower power and efficiency of cutoff"based designs could increase rather than decrease the complexity, duration, or expense of controlled clinical trials" Trochim and $appelleri, 2883, 583!. This means involving more people, more time, and more money. This presents some serious ethical dilemmas. Using a medical example, "if the drug is eventually found safe and effective, more patients will have been denied optimal care in an (. design than in a randomi;ed clinical trial. If the drug is found to have unacceptable side effects for the level of effectiveness, more patients will have been exposed to the risk of side effects in (. design than in a randomi;ed clinical trial. #ither way, more patients will be given the wrong therapy in an (. design than in a randomi;ed clinical trial" -illiams 2<I!. The benefits and drawbacks to (. need to be examined carefully if you choose to use this research design in your evaluation or study. In order to successfully design your evaluation, you must closely examine how ethics and moral decisions complicate the theory of validity. This web paper has been written to complement %ill Trochim&s Enowledge %ase. The format of my discussion follows the general outline of the theory of validity as it is presented in his web site. %y no means is this an exhaustive discussion of where ethical dilemmas can occur in program evaluations. (ather, I prepared this paper with the intention of helping you prepare for the some of the moral issues and decisions you will have to make as you stage your program evaluation and as you attempt to maintain validity throughout your research. At times it may be difficult, and you will have to compromise between your moral ethics and the research standards you want to adhere to.

As a social science researcher, you will have to translate your personal ethics into your professional ethics, and both codes of ethics should reflect the fact that you are a part of larger society. "The role of the evaluator as member of society at large reflects our presence in a democratic society where common citi;enship with it certain expectations of duty, responsibility and practice" Cewman 2JJ!. 'ocial science research usually intends to contribute beneficial information to society. This concern for the well being of others should be present throughout all stages of your work, "an underlying tenet of -estern democracy is that every citi;en has the responsibility to protect and defend the common good" Cewman 2J3!.

*ction Research
)hat Is *ction Research+
Action (esearch A(! has its academic roots in sociology, social psychology, psychology, organi;ational studies, and education. Action research can be described as a family of research methodologies which pursue action or change! and research or understanding! at the same time. In most of its forms it does this by using a cyclic or spiral process which alternates between action and critical reflection. In the later cycles, it alternates between data collection and interpretation in the light of the understanding developed in the earlier cycles. It is thus an emergent process, which takes shape as understanding increases4 it is an iterative process, which converges towards a better understanding of what happens. In most of its forms it is also participative and 6ualitative. Action (esearch is a methodology, which is intended to have both action outcomes and research outcomes. The action is primary. In distinction, there are some forms of action research where research is the main emphasis and the action is almost a fringe benefit. The responsiveness of action research allows it to be used to develop hypotheses from the data, "on the run" as it were. It can therefore also be used as a research tool for investigative or pilot research, and generally for diagnosis or evaluation. %, .ick, Action (esearch (esources!

*ction Research:
Is educative .eals with individuals as members of social groups

Is problem"focused, context"specific and future"orientated Involves a change intervention Involves a cyclic process in which research, action and evaluation are interlinked Aims at improvement and involvement Is founded on a research relationship in which those involved are participants in the

change process =art # and %ond ) 288M!

3ualitati"e Research Methods

Intro!uction3 This website presents a tutorial on 6ualitative research methods. It is designed to help readers with little or no knowledge on the sub0ect. There are several types and classifications of 6ualitative research methods, but here only five of them are discussed $reswell, 288I!.
A 6ualitative research may be generally defined as a study, which is conducted in a natural setting where the researcher, an instrument of data collection, gathers words or pictures, analy;es them inductively, focuses on the meaning of participants, and describes a process that is both expressive and persuasive in language. $reswell 288I! defines 6ualitative study as,
Qualitative research is an inquiry process of understanding based on distinct methodological traditions of inquiry that explore a social or human problem. The researcher builds a complex, holistic picture, analyzes words, report detailed views of informants, and conducts the study in a natural setting.

Bualitative research should not be viewed as an easy substitute for a OstatisticalP or 6uantitative study. It demands a commitment to an extensive time in the field, engagement in the complex, time"consuming process of data analysis, writing of long passages, and participation in a form of social and human science research that does not have firm guidelines or specific procedures and is evolving and changing constantly. *or reasons why one could conduct 6ualitative research , click here.

Sampling In Research
Mugo Fridah W.

I(TRE!FCTIE(
This tutorial is a discussion on sampling in research it is mainly designed to e6iup beginners with knowledge on the general issues on sampling that is the purpose of sampling in research, dangers of sampling and how to minimi;e them, types of sampling and guides for deciding the sample si;e. *or a clear flow of ideas, a few definitions of the terms used are given.

)hat is research+
According -ebster 28IM!, to research is to search or investigate exhaustively. It is a careful or diligent search, studious in6uiry or examination especially investigation or experimentation aimed at the discovery and interpretation of facts, revision of accepted theories or laws in the light of new facts or practical application of such new or revised theories or laws, it can also be the collection of information about a particular sub0ect.

)hat is a sample+
A sample is a finite part of a statistical population whose properties are studied to gain information about the whole -ebster, 28IM!. -hen dealing with people, it can be defined as a set of respondents people! selected from a larger population for the purpose of a survey. A population is a group of individuals persons, ob0ects, or items from which samples are taken for measurement for example a population of presidents or professors, books or students.

)hat is samplin + Samplin is the act5 process5 or techni2ue of selectin a suitable sample5 or a representati"e part of a population for the purpose of determinin parameters or characteristics of the 4hole population)hat is the purpose of samplin + To dra4 conclusions about populations from samples5 4e must use inferential statistics 4hich enables us to determine a populationIs characteristics by directly obser"in only a portion 8or sample9 of the population- )e obtain a sample rather than a complete enumeration 8a census 9 of the population for many reasons- Eb"iously5 it is cheaper to obser"e a part rather than the 4hole5 but 4e should prepare oursel"es to cope 4ith the dan ers of usin samples- In this tutorial5 4e 4ill in"esti ate "arious 0inds of samplin procedures- Some are better than others but all may yield samples that are inaccurate and unreliable- )e 4ill learn ho4 to minimi6e these dan ers5 but some potential error is the price 4e must pay for the con"enience and sa"in s the samples pro"ideThere would be no need for statistical theory if a census rather than a sample was always used to obtain information about populations. %ut a census may not be practical and is almost never economical. There are six main reasons for sampling instead of doing a census. These are4 "#conomy "Timeliness "The large si;e of many populations "Inaccessibility of some of the population ".estructiveness of the observation "accuracy

The economic ad"anta e of usin a sample in research Eb"iously5 ta0in a sample re2uires fe4er resources than a census- #or e,ample5 let us assume that you are one of the "ery curious students around- .ou ha"e heard so much about the famous Cornell and no4 that you are there5 you 4ant to hear from the insiders- .ou 4ant to 0no4 4hat all the students at Cornell thin0 about the 2uality of teachin they recei"e5 you 0no4 that all the students are different so they are li0ely to ha"e different perceptions and you belie"e you must et all these perceptions so you decide because you 4ant an indepth "ie4 of e"ery student5 you 4ill conduct personal inter"ie4s 4ith each one of them and you 4ant the results in DG days only5 let us assume this particular time you are doin your research Cornell has only DG5GGG students and those 4ho are helpin are so fast at the inter"ie4in art that to ether you can inter"ie4 at least =G students per person per day in addition to your =B credit hours of course 4or0- .ou 4ill re2uire =GG research assistants for DG days and since you are payin them minimum 4a e of JK-GG per hour for ten hours 8JKG-GG9 per person per day5 you 4ill re2uire J=GGGGG-GG 7ust to complete the inter"ie4s5 analysis 4ill 7ust be impossible- .ou may decide to hire additional assistants to help 4ith the analysis at another J=GGGGG-GG and so on assumin you ha"e that amount on your accountAs unrealistic as this example is, it does illustrate the very high cost of census. *or the type of information desired, a small wisely selected sample of $ornell students can serve the purpose. ?ou donQt even have to hire a single assistant. ?ou can complete the interviews and analysis on your own. (arely does a circustance re6uire a census of the population, and even more rarely does one 0ustify the expense.

The time factor* sample may pro"ide you 4ith needed information 2uic0ly- #or e,ample5 you are a !octor and a disease has bro0en out in a "illa e 4ithin your area of 7urisdiction5 the disease is conta ious and it is 0illin 4ithin hours nobody 0no4s 4hat it is- .ou are re2uired to conduct 2uic0 tests to help sa"e the situation- If you try a census of those affected5 they may be lon dead 4hen you arri"e 4ith your results- In such a case 7ust a fe4 of those already infected could be used to pro"ide the re2uired informationThe "ery lar e populations
)any populations about which inferences must be made are 6uite large. *or example, $onsider the population of high school seniors in United 'tates of America, agroup numbering <,JJJ,JJJ. The

responsible agency in the government has to plan for how they will be absorbed into the differnt departments and even the private sector. The employers would like to have specific knowledge about the studentQs plans in order to make compatiple plans to absorb them during the coming year. %ut the big si;e of the population makes it physically impossible to conduct a census. In such a case, selecting a representative sample may be the only way to get the information re6uired from high school seniors.

The partly accessible populations

There are 'ome populations that are so difficult to get access to that only a sample can be used. :ike people in prison, like crashed aeroplanes in the deep seas, presidents e.t.c. The inaccessibility may be economic or time related. :ike a particular study population may be so costly to reach like the population of planets that only a sample can be used. In other cases, a population of some events may be taking too long to occur that only sample information can be relied on. *or example natural disasters like a flood that occurs every 2JJ years or take the example of the flood that occured in CoahQs days. It has never occured again.

The destructi"e nature of the obser"ation Sometimes the "ery act of obser"in the desired charecteristic of a unit of the population destroys it for the intended use- Good e,amples of this occur in 2uality control- #or e,ample to test the 2uality of a fuse5 to determine 4hether it is defecti"e5 it must be destroyed- To obtain a census of the 2uality of a lorry load of fuses5 you ha"e to destroy all of them- This is contrary to the purpose ser"ed by 2uality1control testin - In this case5 only a sample should be used to assess the 2uality of the fuses *ccuracy and samplin * sample may be more accurate than a census- * sloppily conducted census can pro"ide less reliable information than a carefully obtained sample%I*S *(! &RRER I( S*M$LI(G * sample is e,pected to mirror the population from 4hich it comes5 ho4e"er5 there is no uarantee that any sample 4ill be precisely representati"e of the population from 4hich it comes- Chance may dictate that a disproportionate number of untypical obser"ations 4ill be made li0e for the case of testin fuses5 the sample of fuses may consist of more or less faulty fuses than the real population proportion of faulty cases- In practice5 it is rarely 0no4n 4hen a sample is unrepresentati"e and should be discardedSamplin error
-hat can make a sample unrepresentative of its population/ 7ne of the most fre6uent causes is sampling error.

'ampling error comprises the differences between the sample and the population that are due solely to the particular units that happen to have been selected. *or example, suppose that a sample of 2JJ american women are measured and are all found to be taller than six feet. It is very clear even without any statistical prove that this would be a highly unrepresentative sample leading to invalid conclusions. This is a very unlikely occurance because naturally such rare cases are widely distributed among the population. %ut it can occur. :uckily, this is a very obvious error and can be etected very easily. The more dangerous error is the less obvious sampling error against which nature offers very little protection. An example would be like a sample in which the average height is overstated by only one inch or two rather than one foot which is more obvious. It is the unobvious error that is of much concern. There are two basic causes for sampling error. 7ne is chance, That is the error that occurs 0ust because of bad luck. This may result in untypical choices. Unusual units in a population do exist and there is always a possibility that an abnormally large number of them will be chosen. *or example, in a recent study in which I was looking at the number of trees, I selected a sample of households randomly but strange enough, the two households in the whole population, which had the highest number of trees 2J,J2I and N5<M ! were both selected making the sample average higher than it should be. The average with these two extremes removed was I3I trees. The main protection agaisnt this kind of error is to use a large enough sample. The second cause of sampling is sampling bias. 'ampling bias is a tendency to favour the selection of units that have paticular characteristics. 'ampling bias is usually the result of a poor sampling plan. The most notable is the bias of non response when for some reason some units have no chance of appearing in the sample. *or example, take a hypothetical case where a survey was conducted recently by $ornell @raduate school to find out the level of stress that graduate students were going through. A mail 6uestionnaire was sent to 2JJ randomly selected graduate students. 7nly M3 responded and the results were that students were not under strees at that time when the actual case was that it was the highest time of stress for all students except those who were writing their thesis at their own pace. Apparently, this is the group that had the time to respond. The researcher who was conducting the study went back to the 6uestionnaire to find out what the problem was and found that all those who had responded were third and fourth Ah. students. %ias can be very costly and has to be gaurded against as much as possible. *or this case, R3JJJ.JJ had been spent and there were no reliable results in addition, it cost the reseacher his 0ob since his employer thought if he was 6ualified, he should have known that before hand and planned on how to avoid it. A means of selecting the units of analysis must be designed to avoid the more obvious forms of bias. Another example would be where you would like to know the average income of some community and you decide to use the telephone numbers to select a sample of the total population in a locality where only the rich and middle class households have telephone lines. ?ou will end up with high average income which will lead to the wrong policy decisions.

(on samplin error 8measurement error9

The other main cause of unrepresentative samples is non sampling error. This type of error can occur whether a census or a sample is being used. :ike sampling error, non sampling error may either be produced by participants in the statistical study or be an innocent by product of the sampling plans and procedures. A non sampling error is an error that results solely from the manner in which the observations are made. The simplest example of non sampling error is inaccurate measurements due to malfuntioning instruments or poor procedures. *or example, $onsider the observation of human weights. If persons are asked to state their own weights themselves, no two answers will be of e6ual reliability. The people will have weighed themselves on different scales in various states of poor caliberation. An individualQs weight fluctuates diurnally by several pounds, so that the time of weighing will affect the answer. The scale reading will also vary with the personQs state of undress. (esponses therefore will not be of comparable validity unless all persons are weighed under the same circumstances. %iased observations due to inaccurate measurement can be innocent but very devastating. A story is told of a *rench astronomer who once proposed a new theory based on spectroscopic measurements of light emitted by a particular star. -hen his colle6ues discovered that the measuring instrument had been contaminated by cigarette smoke, they re0ected his findings. In surveys of personal characteristics, unintended errors may result from, "The manner in which the response is elicited "The social desirability of the persons surveyed "The purpose of the study "The personal biases of the interviewer or survey writer

The inter4iers effect

Co two interviewers are alike and the same person may provide different answers to different interviewers. The manner in which a 6uestion is formulated can also result in inaccurate responses. Individuals tend to provide false answers to particular 6uestions. *or example, some people want to feel younger or older for some reason known to themselves. If you ask such a person their age in years, it is easier for the idividual 0ust to lie to you by over stating their age by one or more years than it is if you asked which year they were born since it will re6uire a bit of 6uick arithmetic to give a false date and a date of birth will definitely be more accurate.

The respondent effect

(espondents might also give incorrect answers to impress the interviewer. This type of error is the most difficult to prevent because it results from out right deceit on the part of the respondee. An example of this is what I witnessed in my recent study in which I was asking farmers how much mai;e they harvested last year 288M!. In most cases, the men tended to lie by saying a figure which is the reccomended expected yield that is 3M bags per acre. The responses from men looked so uniform that I became suspicious. I compared with the responses of the wives of the these men

and their responses were all different. To decide which one was right, whenever possible I could in a tactful way verify with an older son or daughter. It is important to acknowledge that certain psychological factors induce incorrect responses and great care must be taken to design a study that minimi;es their effect.

/no4in the study purpose

Enowing why a study is being conducted may create incorrect responses. A classic example is the 6uestion, -hat is your income/ If a government agency is asking, a different figure may be provided than the respondent would give on an application for a home mortgage. 7ne way to guard against such bias is to camouflage the studyQs goals4 Another remedy is to make the 6uestions very specific, allowing no room for personal interpretation. *or example, "-here are you employed/" could be followed by "-hat is your salary/" and ".o you have any extra 0obs/" A se6uence of such 6uestions may produce more accurate information.

Induced bias
*inally, it should be noted that the personal pre0udices of either the designer of the study or the data collector may tend to induce bias. In designing a 6uestionnaire, 6uestions may be slanted in such a way that a particular response will be obtained even though it is inacurrate. *or example, an agronomist may apply fertili;er to certain key plots, knowing that they will provide more favourable yields than others. To protect against induced bias, advice of an individual trained in statistics should be sought in the design and someone else aware of search pitfalls should serve in an auditing capacity.

S&L&CTI(G T'& S*M$L&

The preceding section has covered the most common problems associated with statistical studies. The desirability of a sampling procedure depends on both its vulnerability to error and its cost. =owever, economy and reliability are competing ends, because, to reduce error often re6uires an increased expenditure of resources. 7f the two types of statistical errors, only sampling error can be controlled by exercising care in determining the method for choosing the sample. The previous section has shown that sampling error may be due to either bias or chance. The chance component sometimes called random error! exists no matter how carefully the selection procedures are implemented, and the only way to minimi;e chance sampling errors is to select a sufficiently large sample sample si;e is discussed towards the end of this tutorial!. 'ampling bias on the other hand may be minimi;ed by the wise choice of a sampling procedure.

T.$&S E# S*M$L&S
There are three primary kinds of samples, the convenience, the 0udgement sample, and the random sample. They differ in the manner in which the elementary units are chosen.

The con"enient sample

A convenience sample results when the more convenient elementary units are chosen from a population for observation.

The 7ud ement sample

A 0udgement sample is obtained according to the discretion of someone who is familiar with the relevant characteristics of the population.

The random sample

This may be the most important type of sample. A random sample allows a known probability that each elementary unit will be chosen. *or this reason, it is sometimes referred to as a probability sample. This is the type of sampling that is used in lotteries and raffles. *or example, if you want to select 2J players randomly from a population of 2JJ, you can write their names, fold them up, mix them thoroughly then pick ten. In this case, every name had any e6ual chance of being picked. (andom numbers can also be used see :apin page I2!.

T4"ES 56 R&)D5M S&M"7ES

* simple random sample

A simple random sample is obtained by choosing elementary units in search a way that each unit in the population has an e6ual chance of being selected. A simple random sample is free from sampling bias. =owever, using a random number table to choose the elementary units can be cumbersome. If the sample is to be collected by a person untrained in statistics, then instructions may be misinterpreted and selections may be made improperly. Instead of using a least of random numbers, data collection can be simplified by selecting say every 2Jth or 2JJth unit after the first unit has been chosen randomly as discussed below. such a procedure is called systematic random sampling.

* systematic random sample

A systematic random sample is obtained by selecting one unit on a random basis and choosing additional elementary units at evenly spaced intervals until the desired number of units is obtained. *or example, there are 2JJ students in your class. ?ou want a sample of 3J from these 2JJ and you have their names listed on a piece of paper may be in an alphabetical order. If you choose to use systematic random sampling, divide 2JJ by 3J, you will get M. (andomly select any number between 2 and five. 'uppose the number you have picked is <, that will be your starting number. 'o student number < has been selected. *rom there you will select every Mth name until you reach the last one, number one hundred. ?ou will end up with 3J selected students.

* stratified sample
A stratified sample is obtained by independently selecting a separate simple random sample from each population stratum. A population can be divided into different groups may be based on some characteristic or variable like income of education. :ike any body with ten years of education will be in group A, between 2J and 3J group % and between 3J and 5J group $. These groups are referred to as strata. ?ou can then randomly select from each stratum a given number of units which may be based on proportion like if group A has 2JJ persons while group % has MJ, and $ has 5J you may decide you will take 2JS of each. 'o you end up with 2J from group A, M from group % and 5 from group $.

* cluster sample
A cluster sample is obtained by selecting clusters from the population on the basis of simple random sampling. The sample comprises a census of each random cluster selected. *or example, a cluster may be some thing like a village or a school, a state. 'o you decide all the elementary schools in Cewyork 'tate are clusters. ?ou want 3J schools selected. ?ou can use simple or systematic random sampling to select the schools, then every school selected becomes a cluster. If you interest is to interview teachers on thei opinion of some new program which has been introduced, then all the teachers in a cluster must be interviewed. Though very economical cluster sampling is very susceptible to sampling bias. :ike for the above case, you are likely to get similar responses from teachers in one school due to the fact that they interact with one another.

$FR$ES&#FL S*M$LI(G
Aurposeful sampling selects information rich cases for indepth study. 'i;e and specific cases depend on the study purpose. There are about 2N different types of purposeful sampling. They are briefly described below for you to be aware of them. The details can be found in Aatton 288J!Ag 2N8"2IN.

&,treme and de"iant case samplin This in"ol"es learnin from hi hly unusual manifestations of the phenomenon of interest5 suchas outstandin successes5 notable failures5 top of the class5 dropouts5 e,otic e"ents5 crisesIntensity samplin This is information rich cases that manifest the phenomenon intensely5 but not e,tremely5 such as ood students5poor students5 abo"e a"era e/belo4 a"era eMa,imum "ariation samplin This in"ol"es purposefully pic0in a 4ide ran e of "ariation on dimentions of interest- This documents uni2ue or di"erse "ariations that ha"e emer ed in adaptin to different conditions- It also identifies important common patterns that cut across "ariations- Li0e in the e,ample of inter"ie4in Cornell students5 you may 4ant to et students of different nationalities5 professional bac0 rounds5 cultures5 4or0 e,perience and the li0e'omo enious samplin This one reduces "ariation5 simplifies analysis5 facilitates roup inter"ie4in - Li0e instead of ha"in the ma,imum number of nationalities as in the abo"e case of ma,imum "ariation5 it may focus on one nationality say *mericans onlyTypical case samplin It in"ol"es ta0in a sample of 4hat one 4ould call typical5 normal or a"era e for a particular phenomenon5 Stratified purposeful samplin This illustrates charecteristics of particular sub roups of interest and facilitates comparisons bet4een the different roupsCritical case samplin L This permits lo ical enerali6ation and ma,imum application of information to other cases li0e ;If it is true for this one case5 it is li0ely to be true of all other cases- .ou must ha"e heard statements li0e if it happenned to so and so then it can happen to anybody- Er if so and so passed that e,am5 then anybody can passSno4ball or chain samplin This particular one identifies5 cases of interest from people 4ho 0no4 people 4ho 0no4 4hat cases are information rich5 that is ood e,amples for study5 ood inter"ie4 sub7ects- This is commonly used in studies that may be loo0in at issues li0e the homeless households- )hat you do is to et hold of one and he/she 4ill tell you 4here the others are or can be found- )hen you find those others they 4ill tell you 4here you can et more others and the chain continues-

Criterion samplin 'ere5 you set a criteria and pic0 all cases that meet that criteria for e,ample5 all ladies si, feet tall5 all 4hite cars5 all farmers that ha"e planted onions- This method of samplin is "ery stron in 2uality assuranceTheory based or operational construct samplin - #indin manifestations of a theoretical construct of interest so as to elaborate and e,amine the constructConfirmin and disconfirmin cases &laboratin and deepenin initial analysis li0e if you had already started some study5 you are see0in further information or confirmin some emer in issues 4hich are not clear5 see0in e,ceptions and testin "ariationEpportunistic Samplin This in"ol"es follo4in ne4 leads durin field 4or05 ta0in ad"anta e of the une,pected fle,ibilityRandom purposeful samplin This adds credibility 4hen the purposeful sample is lar er than one can handle- Reduces 7ud ement 4ithin a purposeful cate ory- %ut it is not for enerali6ations or representati"enessSamplin politically important cases This type of samplin attracts or a"oids attractin attention undesired attention by purposisefully eliminatin from the sample political cases- These may be indi"iduals5 or localitiesCon"enience samplin It is useful in ettin eneral ideas about the phenomenon of interest- #or e,ample you decide you 4ill inter"ie4 the first ten people you meet tomorro4 mornin - It sa"es time5 money and effort- It is the poorest 4ay of ettin samples5 has the lo4est credibility and yields information1poor casesCombination or mi,ed purposeful samplin This combines "arious samplin strate ies to achie"e the desired sample- This helps in trian ulation5 allo4s for fle,ibility5 and meets multiple interests and needs- )hen selectin a samplin strate y it is necessary that it fits the purpose of the study5 the resources a"ailable5 the 2uestion bein as0ed and the constraints bein faced- This holds true for samplin strate y as 4ell as sample si6eS*M$L& SIM&

%efore deciding how large a sample should be, you have to define your study population. *or example, all children below age three in TomkinQs $ounty. Then determine your sampling frame which could be a list of all the chidren below three as recorded by TomkinQs $ounty. ?ou can then struggle with the sample si;e. The 6uestion of how large a sample should be is a difficult one. 'ample si;e can be determined by various constraints. *or example, the available funding may prespecify the sample si;e. -hen research costs are fixed, a useful rule of thumb is to spent about one half of the total amount for data collection and the other half for data analysis. This constraint influences the sample si;e as well as sample design and data collection procedures. In general, sample si;e depends on the nature of the analysis to be performed, the desired precision of the estimates one wishes to achieve, the kind and number of comparisons that will be made, the number of variables that have to be examined simultaneously and how heterogenous a universe is sampled. *or example, if the key analysis of a randomi;ed experiment consists of computing averages for experimentals and controls in a pro0ect and comparing differences, then a sample under 2JJ might be ade6uate, assuming that other statistical assumptions hold. In non"experimental research, most often, relevant variables have to be controlled statistically because groups differ by factors other than chance. )ore technical considerations suggest that the re6uired sample si;e is a function of the precision of the estimates one wishes to achieve, the variability or variance, one expects to find in the population and the statistical level of confidence one wishes to use. The sample si;e C re6uired to estimate a population mean average! with a given level of precision is, The s6uare root of CT 2.8N!U H!1precision -here H is the population standard deviation of the for the variable whose mean one is interested in estimating. Arecision refers to width of the interval one is willing to tolerate and 2.8N reflects the confidence level. *or details on this please see 'alant and .illman 288<!. *or example, to estimate mean earnings in a population with an accuracy of R2JJ per year, using a 8MS confidence interval and assuming that the standard deviation of earnings in the population is R2NJJ.J, the re6uired sample si;e is 8I5,K 2.8N! 2NJJ12JJ!L s6uared. .eciding on a sample si;e for 6ualitative in6uiry can be even more difficult than 6uantitative because there are no definite rules to be followed. It will depend on what you want to know, the purpose of the in6uiry, what is at stake, what will be usefull, what will have credibility and what can be done with available time and resources. -ith fixed resources which is always the case, you can choose to study one specific phenomenon in depth with a smaller sample si;e or a bigger sample si;e when seeking breadth. In purposeful sampling, the sample should be 0udged on the basis of the purpose and rationale for each study and the sampling strategy used to achieve the studies purpose. The validity, meangfulness, and insights generated from 6ualitative in6uiry have more to do with the information"richness of the cases selected and the observational1analytical capabilities of the researcher than with sample si;e.

CE(CLFSIE(
In conclusion, it can be said that using a sample in research saves mainly on money and time, if a suitable sampling strategy is used, appropriate sample si;e selected and necessary precautions taken to reduce on sampling and measurement errors, then a sample should yield valid and reliable information. .etails on sampling can be obtained from the references included below and many other books on statistics or 6ualitative research which can be found in libraries

Introduction
;It is the *lbert &instein

to
theory that

Research
decides 4hat can

!esi n
be obser"ed-;

#xperiments, if conducted correctly can enable a better understanding of the relationship between a causal hypothesis and a particular phenomenon of theoretical or practical interest. 7ne of the biggest challenges is deciding which research methodology to use. O(esearch that tests the ade6uacy of research methods does not prove which techni6ue is better4 it simply provides evidence relating to the potential strengths and limitations of each approach.P =oward, 28IM!. In research and evaluation, a true experimental design also known as random experimental design!, is the preferred method of research. It provides the highest degree of control over an experiment, enabling the researcher the ability to draw causal inferences with a high degree of confidence. A true experimental design is a design in which sub0ects are randomly assigned to program and control groups. -ith this techni6ue, every member of the target population has an e6ual chance of being selected for the sample. The fact that every member of the target population has an e6ual chance of being selected for the sample makes this design the strongest method for establishing e6uivalence between a program and control group. Buasi"experimental group design differs from true experimental group design by the omission of random assignment of sub0ects to a program and control group. As a result, you can not be sure that the program and the control group are e6uivalent. The use of random experimental design to randomly assign sub0ects to a program and control group, controls for all threats to internal validity. Issues of internal validity arise when groups in the study are none6uivalent. ?our ability as a researcher to say that your treatment caused the effect is compromised. In most causal hypothesis tests, the central inferential 6uestion is whether any observed outcome differences between groups are attributable to the program or instead to some other factor. In order to argue for the internal validity of an inference the analyst must attempt to demonstrate that the program and not some plausible alternative explanation is responsible for the effect. In the literature on internal validity, these plausible alternative explanations or factors are often termed threats" to internal validity" Trochim, 2889!.

:et us consider an instance in which an investigator wishes to determine if a program designed to reduce pre0udice is effective. In this instance, the independent variable is a lecture on pre0udice for grammar school students. *or the dependent measure, the researcher will use a standard self"report test of pre0udice. To conduct the study, the researcher selects a group of students from a local grammar school and administers the pre0udice 6uestionnaire to all of them. A week later, all the students receive the lecture on pre0udice and, after the lecture, again are tested. The next step is to find out whether the pre0udice scores collected before the intervention call them the pretest scores! are substantially higher than scores obtained following the lecture the posttest scores!. The researcher might conclude that, if the posttest responses are lower than the pretest responses, the intervention has reduced sub0ects& pre0udice. As you can see, what the researcher has done is assume that changes in the dependent variable were caused by the introduction of the independent variable. %ut what possibilities other than the operation of the independent variable on the dependent variable might explain the observed relationship $ampbell H 'tanley, 28N5!/ The section on experimental design explains several such threats to internal validity . This is an important point to note. The research designs and methods used in an evaluation have a direct effect on whether or not a program is perceived effective. .id the cause really produced the effect or was it some other plausible explanation/ If the cause produced the effect, can it be generali;ed to a different group in another location/ These are 6uestions of validity. "The first thing we have to ask is, "validity of what/" -hen we think about validity in research, most of us think about research components. -e might say that a measure is a valid one, or that a valid sample was drawn, or that the design had strong validity. All of those statements are technically incorrect. )easures, samples and designs don&t &have& validity "" only propositions can be said to be valid. Technically, we should say that a measure leads to valid conclusions or that a sample enables valid inferences, and so on. It is a proposition, inference or conclusion that can &have& validity" Trochim, 2889!.