8. Scheffé Post-Hoc Comparisons

 

When to use a Scheffé Posthoc comparison

 

Whenever an ANOVA model is used to examine the differences among more than 2 groups, a posthoc procedure can be used to compare differences between all pairs of means. Posthoc comparisons are very similar to t-tests. However, posthoc comparisons are more appropriate for multiple tests, because they help control type-I error.

 

Type I-error is the chance of wrongly accepting differences between means as significant. In a t-test, this chance is controlled to be at most 5%. In other word, we only accept 2 means as significantly different if the p-value in the t-test is less than “.05.” In a study that has several groups, we could do several t-tests to compare all the differences between means. However, each t-test has a separate 5% chance of making a wrong conclusion by falsely declaring 2 means significantly different. When we do several tests, the chance of making at least one wrong conclusion starts to expand dramatically. To understand “why” consider a simple toss of a fair coin. Each toss has a 50% chance of yielding “heads.” What if we tossed the coin 100 times? What is the chance of having at least one “heads?” Pretty darn likely! In the same way, the more t-tests we perform, the more likely we’ll get at least one conclusion wrong. Fortunately, posthoc comparisons have been con structured to adjust for this problem. They are more conservative than t-tests and control for Type-I error.

 

Example 8.1

 

We’ll return once again to the diet example. However, this time there are 3 different diets; including pizza, beer and cream. The outcome is weight gain. The research question follows:

 

 “Is there any difference in weight gain between the 3 diets?”

 

The first step is to create an ANOVA table similar to our previous example. See the previous example for a detailed explanation of how to calculate a Oneway ANOVA. However, a brief hand-worked solution is provided below: