|






 
Reliability and Validity
 — 



r —nything used to pick one person over another
for a job, including:
r Resume
r Interview
r Intelligence test
r Drug test
r Skills test
r —nd a bunch of other things
 —

 
 
 


r — typical selection system might include:
r Review resume
r Phone interview
r Interviews with HR & Supervisor
r Intelligence test
r Drug test
r Background check
 G
 
 
 

  
r ^egal ² no ¶unjustified· adverse impact
r Reliable ² produce consistent results
r Valid ² measure what they are supposed to
measure
r Practical ² cost effective
r —cceptable to the applicant ² lets not make the
applicant mad
 º

 


º






 




r ºorrelation ² compares 2 sets of numbers
(height & weight)
r From ²1.00 to 0.00 to +1.00
r Generally, we are interested in the 0.00 to +1.00
 ë  

 


 !"##
r +1.00 means that the two numbers are perfectly
positively correlated
r If X goes up then Y also goes up
r If X goes down then Y also goes down
r If X stays the same Y also stays the same

r We NEVER see this in the real world.

 ë

 


r w +0.00 to +1.00
r If X goes up then Y goes up
r Statistical significance is strongly influenced by
sample size:
r — sample of <30 is questionable when calculating
correlation
 |






r Degree of dependability, consistency or stability
of scores
r Unreliable ² I got a 32 on the —º in May and
21 in June. Something is wrong.
r Reliable ² get a 32 in May and a 31 in June.
 |


r he test itself
r he situation (hot room)
r he person (sick, bad day)
r Scoring errors
r —dministrative errors in giving a test
 M






  $

M






 
 
r est ² retest
r ake the same test at time1 and time2
r ºorrelate the scores at t1 and t2
r Problems ² time consuming & learning
r Parallel or equivalent forms
r ake forms — and B of same test
r ºorrelate the scores of — & B
r Problems ² time consuming & equivalency of forms
 M






 


M






 
 
r Split-half (could divide test into 2 sets of
Split-
questions, or people into 2 sets, then correlate
Half1 with Half2
r Issues ² need sufficient sample size, and must split
truly randomly, so that halves are equivalent
 M






 


M






 
 
r ‰uder--Richardson & ºronbach·s —lpha
‰uder
r For different types of measures
r Basically ² they calculate the average of all possible
split--halves
split
r Requires a statistical package (SPSS)
 M






 


M






 


r ºompares the agreement between (for example)
3 interviewers
r If we rate on a scale of 1 to 10, 3 raters can
differ, but we should not see a 9, a 5 and a 2 for
the same candidate.
 @



 




r If a selection device is reliable
r It may or may not be valid
r If a selection device is not reliable it cannot be
valid
 i


 
r Definition ² the degree to which available
evidence supports inferences made from scores
on selection devices
r Hi —º scores predict success in college
r Meaning ² does the device measure what it is
supposed to measure?
 i



 


r ºriterion--related validity
ºriterion
r Predictive
r ºoncurrent

r ºontent validity
r ºonstruct validity
 º

%



 
º

%



 






r Give a test to applicants when hired and
compare to performance on the job
r ºorrelate test with performance
r Problems
rSubjectivity of performance measures
r ime lag from test to performance
 º

%



 
º

%



 
 



r Give a test to current employees whose job
performance is already known
r ºorrelate test with performance
r Problems
r —re current employees similar to applicants
º

%



 
º

% 



  G

 &
r Reasonably stable over time
r Must have a bias free criterion
r Must be a representative sample
r Sample must be large enough to be stable
 º



r Based on the content of the job
r Must be representative of the content of the job
r Needs a complete job analysis
r It is a mini-
mini-job sample
r Develop using Subject Matter Experts (SME·s)
 º 



r Measuring an abstract concept such as:
intelligence or sociability
r We may measure intelligence with an IQ test,
but does it really measure the construct
(intelligence)?
r his creates a complex situation to analyze ² lets
leave it at that.
 i


 

'


r —pplying a validity study from one situation (job
or company) to another.
r Burden of proof for applicability is on the user
of the selection device.
 ½  






r Refers generally to cost effectiveness of the

measure
r Depends greatly on the nature of the job
½  




&



r ^ess crucial in a ¶tight· job market
r If I find a drug test insulting will I turn down
your job if I have a second offer that does not
require a drug test?
G M