Demystifying Statistical Analysis 3: The One-Way ANOVA Expressed in Linear Regression
In the previous part of this series, we looked at how the independent t-test can be expressed in a linear regression. The independent t-test is used when comparison is made between 2 unrelated groups, but what happens when we want to compare across more than 2 groups? In such a situation, the one-way ANOVA is the statistical analysis of choice. Just like the independent t-test, the one-way ANOVA involve the use of categorical predictors, and can be expressed in a linear regression.
For those who are unfamiliar with the one-way ANOVA and just want to know how it is usually conducted in SPSS, Laerd Statistics provides a comprehensive step-by-step guide. Otherwise, I will be explaining about the test using the following regression equation, with the help of the textbook “Data Analysis: A Model Comparison Approach” by Carey Ryan, Charles M. Judd, and Gary H. McClelland:
Ŷi = b0 + b1X1i + b2X2i
When comparing more than 2 groups, (m — 1) coded predictors are required in the regression model, where m is the number of groups. In the case of 3 groups, 2 additional parameters b1 and b2 need to be estimated, on top of the intercept b0.
Just as in the independent t-test, dummy coding can be used, but a fixed reference group needs to be selected, where it is coded as 0 for all the constructed predictors. In the case of a categorical predictor that is ordinal (ranked), selecting the lowest ranked or “none” group as the reference group may still make sense. But in the case of a categorical predictor that is nominal (unordered), one must be careful in selecting the reference group, as all the resulting slopes will represent a comparison to it. For example, in a comparison of “Children vs Teens vs Adults” where Children is selected as the reference group, Teens will be coded as 1 and Children/Adults are coded as 0 for predictor X1i, while Adults is coded as 1 and Children/Teens are coded as 0 for predictor X2i. b0 will then represent the Children mean when both X1i and X2i are 0, while b1 represents the difference between Children and Teens, and b2 represents the difference between Children and Adults. The hypothesis tests on these slopes then reveal whether or not the differences are statistically significant.

While contrast coding does not require the selection of a fixed reference group, specific hypotheses need to be constructed to determine what codes to use. For example, in the comparison of “Children vs Teens vs Adults”, one might group Children and Teens together to compare against Adults, and also specifically compare Children against Teens. In such a situation, Children, Teens, Adults will be respectively coded as {-1, -1, 2} for predictor X1i, and {-1, 1, 0} for predictor X2i. For the comparisons to work, these codes must be constructed orthogonally, meaning the sum of the codes in each predictor equal to 0 (-1 + -1 + 2 = 0, -1 + 1 + 0 = 0), and the sum of the multiplication within the group is also equal to 0 (-1*-1 + -1*1 + 2*0 = 0). Like the comparison between 2 groups, b0 represents the mean of the Children, Teen and Adult means. b1 represents 1/3 the difference between Adults vs the mean of Children and Teens (because -1 and 2 are 3 units apart), while b2 represents 1/2 the difference between Children vs Teens (because -1 and 1 are 2 units apart). The hypothesis tests on these slopes also reveal whether or not the differences are statistically significant.

There are many ways that orthogonal contrast codes can be constructed, but the important point to note is that they should always be guided by hypotheses on how the groups may be compared. For example, if the groups Children, Teens and Adults are treated as ordinal, polynomial relationships can be tested. Children, Teens, Adults can be respectively coded as {-1, 0, 1} for predictor X1i (increasing trend), and {-1, 2, -1} for predictor X2i (inverted U-shape trend), to test for linear and quadratic relationships. If b1 is positive and significant, it means that a positive linear relationship exists across the groups; if b2 is positive and significant, it means that a quadratic relationship exists across the groups, where Teens have the highest score on the dependent variable.


Those familiar with one-way ANOVA in statistical packages will notice that the linear regression approach is not exactly equivalent. Rather, the linear regression approach only tests very specific parts of what the one-way ANOVA can produce. This is because the one-way ANOVA is constructing many variations of codes, to test for all the possible comparisons. For this reason, it is always advised that between-group comparisons are corrected to avoid the inflation of Type I errors. Manually, one can divide the p-value of .05 by the number of tests being carried out to make a correction. However, this is a very conservative approach, as the more number of tests being done, the harder it becomes for a test to be statistically significant. In essence, the linear regression approach may be more useful when one has very specific hypotheses to test.
In the subsequent posts of this series, I will continue to explain about other statistical analyses using the same method of linear regression and dummy/contrast coding.
Originally published at: https://learncuriously.wordpress.com/2018/09/09/one-way-anova
