IES 612/STA 4-573/STA 4-576
Spring 2005
Week 7 IES612-week07-lecture.doc
ANOVA MODELS model adequacy
Numeric data samples from t populations obtained
Assume Yij ~ independent N(mi, s2) or Yij = mi + eij with eij ~ independent N(0, s2)
i = 1,2, , t (populations or treatments)
j = 1, 2, , ni (observations)
Residual definition? 
|
Assumption |
Checking? |
Addressing? |
|
Constant
variance? |
Plot eij vs. sample means and look for a pattern |
- Transformation
(e.g. log, sqrt) - Weighted
Least Squares |
|
Normal
responses? |
- Normal
probability plot (normal scores vs. residual quantiles) - Histogram? Boxplot? Stemplot? - 68% of standardized
residuals with -1 and +1 (95% within -2 and +2) |
-
Transformation (sqrt count responses, arcsin-sqrt proportions, log right skewed responses) - GLiMs |
|
Independent? |
Plot
residuals vs. order of observations?
Often implicit part of the design |
Analysis that
reflects dependence? |
|
Outliers? |
Large
standardized residuals? |
Check? Run analysis with and without points? Rank-based methods? |
Why not worry
about quality of fits? Devote a unique
parameter to each group so model specification is not much of an issue.
Example: How about
the Meat study log(bacterial growth) with different
conditions
Consider the adequacy of the model:
Log(Bacterial growth)ij = mi + eij for i=1, 2, 3, 4
(packaging condition) and j=1, 2, 3.
Plot of resid*condition.
Legend: A = 1 obs, B = 2 obs,
etc.
resid
0.4
A A A
0.2 A
A A
A
0.0
A
-0.2 A
A
-0.4
A
A
-0.6
CO2 mixed plastic vacuum
Condition
Plot of resid*yhat. Legend: A = 1 obs,
B = 2 obs, etc.
resid
0.4
A A
A
0.2
A
A A
A
0.0
A
-0.2
A
A
-0.4
A
A
-0.6
3 4 5 6 7 8
yhat
* Variances look constant [know pattern with increasing mean response]
*
None of the residuals stand out and look outlying
Plot of resid*nscore. Legend: A =
1 obs, B = 2 obs, etc.
resid
0.4
B A
0.2 A
A A
A
0.0
A
-0.2 A
A
-0.4
A
A
-0.6
-2 -1 0 1 2
Rank for Variable resid
* normal probability plot looks approximately linear
Example: How about
the Meat study (bacterial growth) with different conditions
Suppose we analyzed bacterial growth directly (i.e. a log-transformation not used)
Consider the adequacy of the model:
(Bacterial growth)ij = mi + eij for i=1, 2, 3, 4 (packaging condition) and j=1, 2, 3.
Plot of resid*condition.
Legend: A = 1 obs, B = 2 obs,
etc.
resid
1000
A
500
A A
A A
0 CA
A
A
-500
A
-1000
CO2 mixed plastic vacuum
Condition
Plot of resid*yhat. Legend: A = 1 obs,
B = 2 obs, etc.
resid
1000
A
500
A A
A A
0 CA
A
A
-500
A
-1000
0
500 1000 1500 2000
yhat
* Variance obviously increases with increasing mean response
Plot of resid*nscore. Legend: A = 1 obs,
B = 2 obs, etc.
resid
1000
A
500
A A
A A
0 A A
A A
A
A
-500
A
-1000
-2 -1 0 1 2
Rank for Variable resid
* Nonlinear pattern here and nonconstant variance goes hand-in-hand
* Transformation in this example fixed both problems
A small digression into designs and a CRD model
|
Gathering data |
||
|
Sampling |
Scientific studies |
Observational studies |
|
-SRS |
-CRD |
-ecological studies |
|
-Stratified RS |
-RCBD |
-natural experiments |
|
-Systematic sampling |
-Latin Squares |
-epidemiology studies |
|
-Cluster |
|
|
|
(more to come Week 11+) |
(more to come Ch 14/2) |
|
CRD = Completely Randomized Design involves randomly assigning treatments to experimental units or randomly sampling from existing populations that you want to compare. Only constraint on randomization is how many experimental units receive a particular treatment.
Treatment = level of a factor in a single factor study OR unique combination of factor levels in a multi-factor study.
Different representations of
one-way model
Cell Means Model: Yij = mi + eij
CRD (treatment effects) Model: Yij = m + ai + eij
In these two models,
m1 = m + a1
m2 = m + a2
mt = m + at
Notice that the Cell Means model has
t unknowns while the CRD model has t+1 unknowns. Thus, we need to constrain the definition of
the treatment effects ai. A common constraint is 
which implies that m is the overall mean of all t
populations. Other constraints could be
considered including a1 =0 OR at =0 OR 
. These
constraints lead to different interpretations of m this is beyond the scope of what we discuss; however, it is
worth realizing that software may vary in the coding that is used. This isnt usually a big worry in anova models since the focus is more on group comparisons
versus estimating the model parameters directly.
H0: m1 = m2= m3= = mt is equivalent to H0: a1 = a2= a3= = at= 0
Nonparametric alternative to the one-way anova model F-test
H0: the distributions all have the same location/center
Ha: at least two distributions differ in terms of location/center
TS. Function of the RANKS of the observations
RR: Special Tables or a normal approximation for large sample sizes
Assumptions: distributions have same SHAPE and same SPREAD [so it isnt a free lunch]
Opinion: Often a useful alternative if outliers present
title Nonparametric ANOVA/ Kruskal-Wallis;
title2 Bacteria in meat data;
data meat;
input
condition $ logcount @@;
count
= exp(logcount);
datalines;
plastic 7.66 plastic 6.98 plastic 7.80
vacuum
5.26 vacuum 5.44 vacuum 5.80
mixed
7.41 mixed 7.33 mixed 7.04
CO2 3.51 CO2 2.91 CO2 3.66
;
options ls=70 nocenter
nodate formdlim=-;
proc npar1way;
title3 response = log(count);
class
condition;
var logcount;
run;
proc npar1way;
title3 response = count;
class
condition;
var count;
run;
(output
edited- NPAR1WAY gives lots of different nonparametric methods)
----------------------------------------------------------------------
Nonparametric ANOVA/ Kruskal-Wallis 1
Bacteria in meat data
response = log(count)
The NPAR1WAY Procedure
Analysis of Variance
for Variable logcount
Classified by Variable condition
condition N Mean
plastic 3 7.480
vacuum 3 5.500
mixed 3 7.260
CO2 3 3.360
Source DF
Sum of Squares Mean Square F Value
Pr > F
Among 3
32.87280 10.957600 94.5844
<.0001
Within 8
0.92680 0.115850
----------------------------------------------------------------------
The NPAR1WAY Procedure
Wilcoxon
Scores (Rank Sums) for Variable logcount
Classified by Variable
condition
Sum of Expected Std Dev Mean
condition N
Scores Under H0 Under H0 Score
plastic 3 30.0 19.50 5.408327 10.0
vacuum 3 15.0 19.50 5.408327 5.0
mixed 3
27.0 19.50 5.408327 9.0
CO2 3 6.0 19.50 5.408327 2.0
Kruskal-Wallis
Test
Chi-Square 9.4615
DF 3
Pr > Chi-Square 0.0237
----------------------------------------------------------------------
Nonparametric ANOVA/ Kruskal-Wallis 7
Bacteria in meat data
response = count
The NPAR1WAY Procedure
Analysis of Variance
for Variable count
Classified by Variable condition
condition N Mean
plastic 3 1879.09259
vacuum 3 251.07441
mixed 3 1439.73191
CO2 3 30.22214
Source DF
Sum of Squares Mean Square F Value
Pr > F
Among 3
7282652.347795 2427550.783 16.5587
0.0009
Within 8
1172820.616230 146602.577
----------------------------------------------------------------------
Nonparametric ANOVA/ Kruskal-Wallis 8
Bacteria in meat data
response = count
The NPAR1WAY Procedure
Wilcoxon
Scores (Rank Sums) for Variable count
Classified by Variable condition
Sum of Expected Std Dev Mean
condition N
Scores Under H0 Under H0 Score
plastic 3
30.0 19.50 5.408327 10.0
vacuum 3 15.0 19.50 5.408327 5.0
mixed 3 27.0 19.50 5.408327 9.0
CO2 3 6.0 19.50 5.408327
2.0
Kruskal-Wallis
Test
Chi-Square 9.4615
DF 3
Pr > Chi-Square 0.0237
----------------------------------------------------------------------
Multiple Comparisons
Suppose you reject the overall hypothesis H0: m1 = m2= m3= = mt
What next? Which means differ?
Linear combination: 
which is estimated by 
. This linear
combination is called a CONTRAST if 
Suppose t=4 populations
Example 1: Contrast for the difference in two means m1 = m2 is 
Example 2: Contrast for comparing the mean of one
population with the average of two other population means 
. This can be
written as a contrast
You can test hypotheses about
contrasts.
H0:
Ha:
TS 
where 
P-value = 
Two contrasts 
and 
are
ORTHOGONAL if 
Treatment SS can be partitioned into t-1 mutually orthogonal contrasts
Example Bacteria growth in meat under different packaging conditions (revisited)
title One-way ANOVA/ CRD example + contrasts;
title2 Bacteria in meat data;
data meat;
input
condition $ logcount @@;
datalines;
plastic 7.66 plastic 6.98 plastic 7.80
vacuum
5.26 vacuum 5.44 vacuum 5.80
mixed
7.41 mixed 7.33 mixed 7.04
CO2 3.51 CO2 2.91 CO2 3.66
;
/*
ORDER=DATA says plastic
< vacuum < mixed < CO2 in
Labels
otherwise sorts
*/
proc glm data=meat order=data;
class
condition;
model
logcount=condition;
output
out=new p=yhat r=resid;
contrast
'plastic vs. rest' condition 3 -1 -1 -1;
estimate
'plastic vs. rest' condition 3 -1 -1 -1;
contrast
'CO2 vs. plastic' condition -1 0 0 1;
estimate
'CO2 vs. plastic' condition -1 0 0 1;
contrast
'CO2 vs. vacuum' condition 0 -1 0
1;
estimate
'CO2 vs. vacuum' condition 0 -1 0
1;
contrast
'CO2 vs. mixed' condition 0 0 -1 1;
estimate
'CO2 vs. mixed' condition 0 0 -1 1;
run;
* output
(edited)
The GLM Procedure
Class Level Information
Class Levels Values
condition 4
plastic vacuum mixed CO2
Number of observations 1
The GLM Procedure
Dependent Variable: logcount
Sum of
Source DF Squares Mean Square F Value
Pr > F
Model 3 32.87280000 10.95760000 94.58
<.0001
Error 8 0.92680000 0.11585000
Corrected Total 11 33.79960000
R-Square Coeff Var Root
MSE logcount
Mean
0.972580 5.768940 0.340367 5.900000
Source DF Type I
condition 3 32.87280000 10.95760000 94.58
<.0001
Source
condition 3 32.87280000 10.95760000 94.58
<.0001
Contrast
plastic vs. rest 1 9.98560000 9.98560000 86.19
<.0001
CO2 vs. plastic 1 25.46160000 25.46160000 219.78
<.0001
CO2 vs. vacuum 1 6.86940000 6.86940000 59.30
<.0001
CO2 vs. mixed 1 22.81500000 22.81500000 196.94 <.0001
Dependent Variable: logcount
Standard
Parameter Estimate Error t Value
Pr > |t|
plastic vs. rest 6.32000000 0.68073490 9.28
<.0001
CO2 vs. plastic -4.12000000 0.27790886 -14.83
<.0001
CO2 vs. vacuum -2.14000000 0.27790886 -7.70
<.0001
CO2 vs. mixed -3.90000000 0.27790886 -14.03 <.0001
Individual versus Experimentwise Error Rates
aI = Individual Type I error rate = Pr(Type I Error on a PARTICULAR comparison)
aE =Experimentwise Type I error rate = Pr(at least one Type I error when conducting a collection of comparisons)
If you have m independent comparisons, aE = 1 (1- aI)m
So, aE increases with the number of comparisons.
Bonferroni inequality: aE ≤ m aI so
set aI = aE /m for individual tests to protect overall at fixed aE
|
Procedure |
|
|
|
Bonferroni |
t-based with Bonferroni adjustment for aI |
|
|
Fishers LSD |
t-based (protected if require overall F to reject first) |
-MCA |
|
Tukeys HSD |
Based on |
-MCA -Can also form simultaneous CIs - harmonic mean often substituted if ni not same |
|
Dunnetts Procedure |
Comparing all groups to control (i.e. t-1 comparisons) |
-MCC |
|
Scheffe S |
General procedure for comparing all possible contrasts |
|
Example: Meat Study Revisited
title One-way
ANOVA/ CRD example + contrasts;
title2 Bacteria in meat data;
data meat;
input
condition $ logcount @@;
datalines;
plastic 7.66 plastic 6.98 plastic 7.80
vacuum
5.26 vacuum 5.44 vacuum 5.80
mixed
7.41 mixed 7.33 mixed 7.04
CO2 3.51 CO2 2.91 CO2 3.66
;
/*
ORDER=DATA says plastic
< vacuum < mixed < CO2 in
Labels otherwise
sorts
*/
proc glm data=meat order=data;
class
condition;
model
logcount=condition;
lsmeans condition / stderr pdiff;
means condition / lsd clm; * Fisher LSD;
means condition / bon scheffe tukey; * Bonferroni, Scheffe, Tukey;
means condition /
bon tukey cldiff; * pairwise CIs generated;
run;
The GLM Procedure
Least Squares Means
logcount
Standard LSMEAN
condition LSMEAN Error Pr > |t| Number
plastic 7.48000000 0.19651124 <.0001 1
vacuum 5.50000000 0.19651124 <.0001 2
mixed 7.26000000 0.19651124 <.0001 3
CO2 3.36000000 0.19651124 <.0001 4
Least Squares Means for effect condition
Pr > |t| for H0: LSMean(i)=LSMean(j)
Dependent Variable: logcount
i/j 1 2 3 4
1 <.0001 0.4514
<.0001
2
<.0001
0.0002 <.0001
3
0.4514 0.0002 <.0001
4
<.0001 <.0001 <.0001
NOTE: To ensure overall protection level, only probabilities associated with pre-planned comparisons should be used
t Confidence Intervals for logcount
Alpha 0.05
Error Degrees of
Freedom 8
Error Mean Square 0.11585
Critical Value of t 2.30600
Half Width of Confidence
Interval 0.453156
95%
Confidence
condition N Mean Limits
plastic 3 7.4800 7.0268 7.9332
mixed 3 7.2600 6.8068 7.7132
vacuum 3 5.5000 5.0468 5.9532
CO2 3 3.3600 2.9068 3.8132
means condition / bon scheffe tukey;
Tukey's Studentized Range
NOTE: This test controls
the Type I experimentwise error rate, but it
generally has a higher Type II error rate than REGWQ.
Alpha 0.05
Error Degrees of
Freedom 8
Error Mean Square 0.11585
Critical Value of Studentized Range 4.52880
Minimum Significant
Difference 0.89
Means with the same letter
are not significantly different.
Mean N
condition
A 7.4800 3
plastic
A
A 7.2600 3
mixed
B 5.5000 3 vacuum
C 3.3600 3 CO2
Bonferroni (Dunn) t Tests for logcount
NOTE: This test controls
the Type I experimentwise error rate, but it
generally has a higher Type II error rate than REGWQ.
Alpha 0.05
Error Degrees of
Freedom 8
Error Mean Square 0.11585
Critical Value of t 3.47888
Minimum Significant
Difference 0.9668
Means with the same letter
are not significantly different.
Mean N
condition
A 7.4800 3
plastic
A
A 7.2600 3
mixed
B 5.5000 3
vacuum
C 3.3600 3 CO2
Scheffe's Test for logcount
NOTE: This test controls
the Type I experimentwise error rate.
Alpha 0.05
Error Degrees of
Freedom 8
Error Mean Square 0.11585
Critical Value of F 4.06618
Minimum Significant
Difference 0.9706
Means with the same letter
are not significantly different.
Mean
N condition
A 7.4800 3
plastic
A
A 7.2600 3 mixed
B 5.5000 3
vacuum
C 3.3600 3
CO
means condition / bon tukey cldiff;
Tukey's Studentized Range
NOTE: This test controls
the Type I experimentwise error rate.
Alpha 0.05
Error Degrees of
Freedom 8
Error Mean Square 0.11585
Critical Value of Studentized Range 4.52880
Minimum Significant
Difference 0.89
Comparisons significant at
the 0.05 level are indicated by ***.
Difference
condition Between Simultaneous 95%
Comparison Means Confidence Limits
plastic - mixed 0.2200 -0.6700
1.1100
plastic - vacuum 1.9800 1.0900
2.8700 ***
plastic - CO2 4.1200 3.2300
5.0100 ***
mixed - plastic -0.2200 -1.1100
0.6700
mixed - vacuum 1.7600 0.8700
2.6500 ***
mixed - CO2 3.9000 3.0100
4.7900 ***
vacuum - plastic
-1.9800 -2.8700 -1.0900
***
vacuum - mixed
-1.7600 -2.6500 -0.8700
***
vacuum - CO2
2.1400 1.2500 3.0300
***
CO2 - plastic -4.1200
-5.0100 -3.2300 ***
CO2 - mixed -3.9000 -4.7900 -3.0100 ***
CO2 - vacuum -2.1400 -3.0300 -1.2500 **
Bonferroni (Dunn) t Tests for logcount
NOTE: This test controls
the Type I experimentwise error rate, but it generally
has a higher Type II error rate than Tukey's for all pairwise comparisons.
Alpha 0.05
Error Degrees of
Freedom 8
Error Mean Square 0.11585
Critical Value of t 3.47888
Minimum Significant
Difference 0.9668
Comparisons significant at
the 0.05 level are indicated by ***.
Difference
condition Between Simultaneous 95%
Comparison Means Confidence Limits
plastic - mixed 0.2200 -0.7468
1.1868
plastic - vacuum 1.9800 1.0132
2.9468 ***
plastic - CO2 4.1200 3.1532
5.0868 ***
mixed - plastic -0.2200 -1.1868
0.7468
mixed - vacuum 1.7600 0.7932
2.7268 ***
mixed - CO2 3.9000 2.9332
4.8668 ***
vacuum - plastic
-1.9800 -2.9468 -1.0132
***
vacuum - mixed
-1.7600 -2.7268 -0.7932
***
vacuum - CO2
2.1400 1.1732 3.1068
***
CO2 - plastic -4.1200 -5.0868 -3.1532 ***
CO2 - mixed -3.9000 -4.8668 -2.9332 ***
CO2 - vacuum -2.1400 -3.1068 -1.1732 ***