Examining differences between more than two groups

When we want to examine differences between two independent groups, a t-test or Man-Whitney U test are the most appropriate to use. If we have more than two independent groups, however, it is not appropriate to use these tests. Instead, we should use an Analysis of Variance (ANOVA) test or one of the non-parametric alternatives.

Assumptions of One-Way ANOVA

To use an ANOVA, our data must meet the following assumptions:

  • The groups are independent (and do not include repeated measures)
  • There is a normal distribution of values in each group
  • The variance in each group is equal (this is call homogeneity of variance)
  • The data is continuous

Description of sample data

In this tutorial, we will be looking at essays written by individuals at three different levels of lexical proficiency (Beginner, Intermediate, and High). We will also be looking at the average meaningfulness score for all words in each text. Meaningfulness scores are one way of determining how many concepts a particular word might be related to (e.g., the word “food” would be more “meaningful” than “zygote” because it could be used in more semantic contexts). We will be determining whether there are any differences in average meaningfulness scores based on lexical proficiency group.

mydata <- read.csv("data/anova_sample.csv", header = TRUE)
summary(mydata)
##   filenames         Proficiency        Frequent_Trigram_Proportion
##  Length:240         Length:240         Min.   :0.0480             
##  Class :character   Class :character   1st Qu.:0.1390             
##  Mode  :character   Mode  :character   Median :0.1931             
##                                        Mean   :0.1930             
##                                        3rd Qu.:0.2427             
##                                        Max.   :0.3623             
##  Meaningfulness_AW Register_Range_CW L1_Word_Reaction_Time Concreteness_AW
##  Min.   :325.4     Min.   :10.23     Min.   :594.3         Min.   :2.143  
##  1st Qu.:363.7     1st Qu.:13.13     1st Qu.:610.5         1st Qu.:2.491  
##  Median :375.4     Median :13.57     Median :615.9         Median :2.619  
##  Mean   :376.1     Mean   :13.52     Mean   :616.4         Mean   :2.644  
##  3rd Qu.:386.3     3rd Qu.:13.93     3rd Qu.:621.4         3rd Qu.:2.772  
##  Max.   :453.7     Max.   :14.86     Max.   :655.6         Max.   :3.539

Checking assumptions, visualizing the data

Checking the assumption of normality

First, we can create density plots to look at the distribution of each group. These density plots suggest that the data is roughly normal and that the variance is roughly equal.

library(ggplot2)
library(viridis) #color-friendly palettes

g1<- ggplot(mydata, aes(x=Meaningfulness_AW, fill=Proficiency)) +
  geom_density(alpha = 0.4) + # "alpha" sets the transparency level
  scale_fill_viridis(discrete = TRUE)

#print(g1)

Density plots for 'Meaningfulness_AW' across proficiency levels: purple for beginner, yellow for intermediate, and blue for advanced, with transparency indicating overlaps. We can also run a Shapiro-Wilk test for each group to test for normality. To do so, we create a dataframe for each proficiency group using dplyr(), then run the Shapiro-Wilk test for each group.

#load dplyr package, which helps us manipulate datasets:
library(dplyr)

#create a new dataframe that includes only Beginner:
beginner.ds <- mydata %>% filter(Proficiency == "Beginner")
#create a new dataframe that includes only Int:
intermediate.ds <- mydata %>% filter(Proficiency == "Int")
#create a new dataframe that includes only High:
high.ds <- mydata %>% filter(Proficiency == "High")

#Test normality for Meaningfulness_AW in Beginner essays
shapiro.test(beginner.ds$Meaningfulness_AW) #p = 0.2336
## 
##  Shapiro-Wilk normality test
## 
## data:  beginner.ds$Meaningfulness_AW
## W = 0.97457, p-value = 0.2336
#Test normality for Meaningfulness_AW in Int essays
shapiro.test(intermediate.ds$Meaningfulness_AW) #p = 0.0112
## 
##  Shapiro-Wilk normality test
## 
## data:  intermediate.ds$Meaningfulness_AW
## W = 0.96912, p-value = 0.0112
#Test normality for Meaningfulness_AW in High essays
shapiro.test(high.ds$Meaningfulness_AW) #p = 0.1591
## 
##  Shapiro-Wilk normality test
## 
## data:  high.ds$Meaningfulness_AW
## W = 0.97366, p-value = 0.1591

The results indicate that Meaningfulness_AW scores Beginner and High essays do not violate the assumption of normality, while Meaningfulness_AW scores for Int essays do (p = 0.011). As mentioned in previous tutorials, the Shapiro-Wilk test is rather stringent, so we can decide whether we think it is appropriate to use a parametric or non-parametric test based on the visual inspection and on the Shapiro-Wilk test results.

Checking the assumption of equal variance

Second, we will check the assumption of equal variance (also known as homogeneity of variance). First, we will generate some boxplots, which (along with the distribution plots above), help give an indication of whether our groups have roughly equal variance.

custom_colors <- c("#377eb8", "orange", "gold1") #for color-friendly option

g2 <- ggplot(data = mydata, aes(x = Proficiency, y = Meaningfulness_AW, fill = Proficiency)) +
  geom_boxplot() +
  scale_fill_manual(values = custom_colors) 

#print(g2)