How To Compare Two Groups in a Within-Subjects Study Design

How to estimate the sample size, perform a paired t-test and plot the data in Python

Statistical Power Analysis with G*Power. Source: Author

Using statistical tests, we can compare different groups to see if there is a significant difference between them.

For example, we might have developed a new version of an app (version A) and we want to see if this update is significantly better than the old one (version B).

For each version, we must measure a so-called dependent variable. The dependent variable could be the required time to perform a specific action on the app, or it could be a score from a usability questionnaire.

If we want to keep the necessary number of participants in our study as low as possible, we should consider a within-subjects study design.

The remainder of this article will focus on the following topics:

• Learn what a within-subjects design is
• Estimate the required study sample size using G*Power
• Perform the paired t-test using scipy.stats
• Plot the data using matplotlib.pyplot

What is within-subjects and between-subjects?