RSE vs R²

The assigned chapter was heavy and fast on the statistical notation as it guided us through linear regression. I can’t say that I found it easy to digest, but I do appreciate how it began with a framing of the topic through the seven questions (Is there a relationship between the two? How strong is it?, etc.). I like this because it gives us more direct understanding of the basics of linear regression and how it can be useful. Further, and more so to the point of this post, it speaks on a higher level to the previous content we have learned in class: Do we understand the question? Are we making it a point to understand the data?

After blasting through the commonplace terms and concepts in the earlier parts of the reading, in 3.1.3 the authors arrive at assessment of the accuracy of a linear regression model. Two significant concepts mentioned in this section are the residual standard error (RSE) and the R-squared (R²), which is also known as the coefficient of determination (Frost, 2013). Both are measures of linear regression fit. What brought my attention to these concepts was not so much my general unfamiliarity with them, but more so how the authors approached them. They provided two different ways of measuring the fit of the model, explaining both the limitations and benefits of each measure. I thought this fit in well with the concept of understanding our data and question on a wider level. There will always be ‘best practice’ when it comes to anything, statistics included, but if we better understand the tools and when to use them we will be able to foster better research and understanding.

All of this considered, I decided to further investigate the benefits of and differences between RSE and R² with the goal of more accurately identifying the situations in which one is better to use than the other.

RSE

RSE is a measure of lack of fit of the model to the data at hand. In simplest terms, from the authors, if the RSE value is very close to to the actual outcome value, then your model fits the data well. If there is a large difference between the values, then the model does not fit the data well (James, Witten, Hastie, & Tibshirani, 2013).

A downside to RSE is that it is very contextual to your data set and thus requires a general understanding of the measurements and overall scale to understand. For example, if the RSE value was 8.0 and the actual outcome was 10.0, without any additional information, this could either be a massive difference or a very small one. What are the units? What’s the overall scale? Admittedly, as a researcher you are going to know this information, but even then, you need to understand the data at the highest level to know the significance of that difference. More so, the point of this example was that it does not make interpretation of the data immediately easy for anyone else.

I did find it surprisingly difficult to find further information on RSE outside of the reading. It seems like it is more or less an R exclusive term and doesn’t have a lot of additional non-technical information.

R²

R² is a different measure of fit that always takes the form of a value between 0 and 1, which means it is independent of the measures of your data. In short, an R² value that is closer to 1 indicates that the regression explains a large amount of the variability, while a value closer to 0 indicates that it did not. This standard of 0–1 is a huge benefit because it is easy to interpret in-the-moment. “In general, the higher the R-squared, the better the model fits your data” (Frost, 2013). However, even R² requires context, because it is difficult to know what a good R² is overall (James, Witten, Hastie, & Tibshirani, 2013).

If you look online, you’ll see some people note that an R² anywhere under .6/.7 is no good at all while others would say anyone using .8 is probably guilty of overfitting (“RMSE vs Coefficient of Determination”, 2012). Further, R² measurements vary across fields of study. Psychology will have naturally lower R² values due to the nature of human subjects. R² does also not indicate whether the regression model is accurate, because you can end up with a low R² for a good model or a high R² for a bad model. Overall, a low R² is not necessarily bad, and a high R² is not necessarily good (Frost, 2013).

So while it might be exciting to have a high R² value, it is essential that you understand it within context of your data, lest you use it as a major analysis point (I would call this akin to over-reliance on the p-value).

All in all, I might very cautiously posit that, in regards to interpretation, RSE might be better for the researchers and R² might be better for readers (which is obviously important, as we have read many times before). But the overall message speaks higher than to just this reading on linear regression: make efforts to understand your data and questions!

References

Frost, J. (2013). Regression Analysis: How Do I Interpret R-squared and Assess the Goodness-of-Fit? The Minitab Blog. http://blog.minitab.com/blog/adventures-in-statistics-2/regression-analysis-how-do-i-interpret-r-squared-and-assess-the-goodness-of-fit

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 6). New York: springer.

RMSE vs Coefficient of Determination (2012). Stack exchange. http://stats.stackexchange.com/questions/38631/rmse-vs-coefficient-of-determination