Beruflich Dokumente
Kultur Dokumente
Statistical Inference
ICTCM 2011 1
Joint Work with
Motivation/Goals
Examples
Binomial process, randomized experiment- binary,
randomized experiment - quantitative response
Series of lab assignments
Discussion points
Student feedback, Evaluation results
Design principles & implementation
Observations, Open questions
ICTCM 2011 3
Motivation
ICTCM 2011 4
Goals
ICTCM 2011 5
Brief overview of labs
Case-study focus
Pre-lab
Background, Review questions submitted in advance
50-minute (computer) lab period
Online instructions
Directed questions following statistical process
Embedded applets or statistical software
Application/Extension
Lab report with partner
ICTCM 2011 6
Example 1: Friend or Foe
(Helper/Hinderer)
Videos
Research question
Pre-lab
Descriptive analysis
Introduction of null hypothesis,
p-value terminology
Plausible values
Conclusions
ICTCM 2011 7
Discussion Points
ICTCM 2011 8
<<After tactile simulation>> How many infants
would need to choose the helper toy for you to
be convinced the choice was not made “at
random,” but they actually prefer the helper toy?
Many students can reason inferentially
“If a choice is made at complete random, then
having 13 infants would be highly unlikely”
“Based on the coin flipping experiment, the results
stated that at/over 12 was extremely rare.
Therefore, at least 12 infants …
“Would be around 12-16 because it seems highly
unlikely that given a 50-50 option 12-16 would
choose the helper toy”
ICTCM 2011 9
<<After tactile simulation>> How many infants
would need to choose the helper toy for you to
be convinced the choice was not made “at
random,” but they actually prefer the helper toy?
But maybe not as well “distributionally”
Is it unusual? = “barely over half”
vs. unusual compared to distribution
Examine language carefully
“Unlikely that choice is random”
“Prove”
“Simulate”, “Repeated this study”
“At random” = 50/50, “model”
“Random” = anything is possible
ICTCM 2011 10
Discussion Points
ICTCM 2011 11
Discussion Points
Tactile simulation
One coin 16 times vs. 16 coins
Population vs process
Defining the parameter
3Ss: statistic, simulate, strength of evidence
“could have been” distribution of data
“what if the null was true” distribution of statistic
Fill in the blank wording
Timing of final report
Follow-up in-class discussion
ICTCM 2011 12
Example 2: Two Proportions
Is Yawning Contagious?
Modelling entire process: data collection,
descriptive statistics, inferential analysis,
conclusions
Parallelisms to first example
Could random assignment alone produce a
difference in the group proportions at least this
extreme?
Card shuffling, recreate two-way table
Extend to own data
ICTCM 2011 13
Lab Instructions
ICTCM 2011 14
Exam Questions
Horizontal axis
Shade p-value
Make up a research question
ICTCM 2011 15
Discussion Points
ICTCM 2011 16
Example 3: Two means
ICTCM 2011 18
Post-Lab Assessment (Fall 2010)
Following the lab comparing two groups on a
quantitative variable (65 responses)
Discuss the purpose of the simulation process
What information does the simulation process reveal
to help you answer the research question?
Essentially correct: 35.4% demonstrated
understanding of the big picture (looking at
repeated shuffles to assess whether the
observed results happened by chance)
Partially: 38.5% (one of null or comparison)
Incorrect: 26.1% (“better understand the data”)
ICTCM 2011 19
Post-Lab Assessment (Fall 2010)
Did students address the null hypothesis?
33.9% E/ 38.5% P/ 27.7% I
Did students reference the random assignment?
36.9% E/ 36.9% P/ 26.2% I
Did students focus on comparing the observed
result?
64.6% E/ 13.8% P/ 21.5% I
Did students explain how they would link the
pieces together and draw their conclusion?
24.6% E/ 60% P/ 15% I
ICTCM 2011 20
Student Surveys
ICTCM 2011 21
Student Surveys
ICTCM 2011 22
Student Surveys
Example 3 simulation
ICTCM 2011 23
Student Surveys
ICTCM 2011 24
Student Surveys
ICTCM 2011 25
Student Surveys
ICTCM 2011 26
Student Surveys
ICTCM 2011 27
Remainder of labs
Lab 4: Random babies
Lab 5: Reese’s Pieces (demo)
Normal approximation, CLT for binary
Transition to formal test of significance (6 steps)
Lab 6: Sleepless nights (finite population)
t approximation, CLT for quantitative, conf interval
Lab 7: Simulation of matched-pairs
Lab 8: Simulation of regression sampling
Chi-square, ANOVA
ICTCM 2011 28
Lab Report
ICTCM 2011 29
Student Feedback (Winter 2011)
ICTCM 2011 30
Student end-of-course surveys (W
11)
ICTCM 2011 31
Student end-of-course surveys
ICTCM 2011 32
Student end-of-course surveys
ICTCM 2011 33
Student end-of-course surveys
ICTCM 2011 34
Student end-of-course surveys
ICTCM 2011 35
Student end-of-course surveys
ICTCM 2011 36
Student end-of-course surveys
ICTCM 2011 37
Student end-of-course surveys
ICTCM 2011 38
Student end-of-course surveys
ICTCM 2011 39
Student end-of-course surveys
ICTCM 2011 40
Student end-of-course surveys
ICTCM 2011 41
Student end-of-course surveys
ICTCM 2011 42
Student end-of-course surveys
ICTCM 2011 43
Student end-of-course surveys
ICTCM 2011 44
Student end-of-course surveys
ICTCM 2011 45
Top 2 most interesting labs
Instructor A
Is Yawning Contagious?
Heart Rates (matched pairs)
Instructor B
Friend or Foe
Is Yawning Contagious?
Reese’s Pieces
ICTCM 2011 46
Top 2 most/least helpful labs
Most helpful:
Friend or Foe
ICTCM 2011 47
Exam 1
In a recent Gallup survey of 500 randomly
selected US adult Republicans, 390 said they
believe their congressional representative
should vote to repeal the Healthcare Law.
Suppose we wish to determine if significantly
more than three-quarters (75%) of US adult
Republicans favor repeal.
The coin tossing simulation applet was used to
generate the following two dotplots (A) and (B).
Which, if either, of the two plots (A) and (B) was
created using the correct procedure? Explain
how you know.
ICTCM 2011 48
Exam 1
ICTCM 2011 49
Exam 2
Heights of females are known to follow a normal
distribution with a mean of 64 inches and a
standard deviation of 3 inches. Consider the
behavior of sample means. Each of the graphs
below depicts the behavior of the sample mean
heights of females.
a. One graph shows the distribution of sample
means for many, many samples of size 10. The
other graph shows the distribution of sample
means for many, many samples of size 50.
Which graph goes with which sample size?
ICTCM 2011 50
Exam 2
ICTCM 2011 51
Exam 2
ICTCM 2011 52
Exam 2
77% picked B
Mixture of appealing to smaller SD/outliers, larger
sample size means smaller p-value, and thinking
in terms of test statistic
A few choices not internally consistent
ICTCM 2011 53
Student understanding of p-value
CAOS questions (final exam)
Statistically significant results correspond to small
p-values
Traditional (National/Hope/CP): 69/86/41%
Randomization (Hope/CP): 95%/95%
Recognize valid p-value interpretation
Traditional (National/Hope/CP): 57/41/74%
Randomization (Hope/CP): 60/72%
p-value as probability of Ho - Invalid
Traditional (National/Hope/CP): 59/69/68%
Randomization (Hope/CP): 80%/89%
ICTCM 2011 54
Student understanding of p-value
ICTCM 2011 55
Student understanding of p-value
ICTCM 2011 56
Student understanding of process
Video game question (Final exam: NCSU, Hope,
Cal Poly, UCLA, Rhodes College)
What is the explanation for the process the
student followed?
Which of the following was used as a basis for
simulating the data 1000 times?
What does the histogram tell you about whether
$5 incentives are effective in improving
performance on the video game?
Which of the following could be the approximate
p-value in this situation?
ICTCM 2011 57
Student understanding of process
Simulation process
Fall: over 40% chose “This process allows her to
determine how many times she needs to replicate
the experiment for valid results.”
About 70% pick “The $5 incentive and verbal
encouragement are equally effective at improving
performance.” as underlying assumption
Still evidence some look at center at zero or
shape as evidence of no treatment effect
1/3 to ½ could estimate p-value from graph
ICTCM 2011 58
Example – 2009 AP Statistics Exam
ICTCM 2011 59
Design Principles
Tactile simulation
Visual, contextual animation of tactile simulation
Intermediate animation capability
Level of student construction
Ease of changing inputs
Connect elements between graphs
Carefully designed, spiraling activities
“Stop!”
Thought questions
Allow for student exploration
ICTCM 2011 60
Implementation
Early in course
Repetition through course, connections
Normal approximations
Lab assignments
Focus on entire statistical process
Motivating research question
Follow-up application
Thought questions
Screen captures
Pre-lab questions
Minitab demos (Adobe Captivate)
Exam questions
ICTCM 2011 61
Observations
ICTCM 2011 63
Suggestions/Open Questions
Scaffolding
Observational units, variable
How would you add one more dot to graph?
At some point, require students to enter the
correct “observed result” (e.g., Captivate)
At some point, ask students to design the
simulation?
Start with fill in the blank interpretation?
ICTCM 2011 64
Suggestions/Open Questions
ICTCM 2011 65
Summary
ICTCM 2011 66