Lift Weights With Your AI Trainer

Less Pain, More Gain — A Tutorial

Weightlifting is a popular form of exercise that can help you lose fat, increase your strength and muscle tone, and improve your bone density. You don’t have to be a bodybuilder or professional athlete to reap the benefits of weightlifting, but you do need to be aware that if done incorrectly, weightlifting carries a risk of injury, especially if you don’t use the correct form.

Artificial Intelligence (AI) has become ubiquitous in our daily lives, even if we don’t realize it yet. I know, I know, AI will not lift weights for you but, how can it help make your workouts better?

If you were to venture into the weightlifting area of your gym, you would probably find people filming themselves as they perform an exercise. This allows them to observe their posture afterwards and, if necessary, correct their form for future repetitions. A correct form is important because it ensures that the exercise targets the correct muscles and also reduces the risk of injury.

In this article I would like to present a use case of pose estimation, a technology we are very familiar with here at Sngular, applied to promoting correct technique in weightlifting. It will include a tutorial on how to get feedback on your front squat form from your video recording using Python and the popular pose estimation model blaze pose.

What Is Pose Estimation?

Human Pose Estimation, or HPE, is a computer vision task designed to recognise and accurately track specific points on the human body. These points are called “keypoints”.

With HPE models, we can extract these points in real time, analyze movement patterns and make further decisions based on this input. This can have multiple applications such as body motion detection, posture correction applications, exercise monitoring or AI fitness coaching.

Movenet sample videos

There are many HPE models, such as MoveNet, YOLOv8-Pose, OpenPose, DeepPose, and Mediapipe-blazepose. Some of them yield the keypoints’ coordinates in the 3D world and others in 2D. Some of them provide key point coordinates in the 3D world, others in 2D. For a more detailed explanation of pose estimation, see this other article.

Tutorial: Build Your Own AI Trainer

In our upcoming tutorial, we will delve into the fascinating world of pose estimation for form correction, with a specific focus on perfecting the front squat movement. This tutorial is designed to address a common issue where individuals often struggle to maintain proper form during front squats and overcharge the lower back.

We will use the ai-trainer repository to give you feedback on your performance during the exercise. This library employs the Mediapipe-blazepose model for detecting the body keypoints. This model yields 33 body pose keypoints (see figure below) in both 2D image coordinates and 3D world coordinates.

Mediapipe Blazepose keypoints format. — by the author

Step 1: Clone the ai-trainer repository and install dependencies

On the terminal, run:

git clone https://github.com/LoboaTeresa/AI-Trainer.git
cd ai_trainer
pip install -r requirements.txt

Step 2: Load the model

Our experimental environment will be a colab notebook. In the first cell run:

from ai_trainer.models import BlazePoseModel

BLAZEPOSE_MODEL_PATH = "./models/blazepose_full.onnx"
blazepose_model = BlazePoseModel(model_path=BLAZEPOSE_MODEL_PATH)

Step 3: Define “good form” heuristics

When performing a squat, it is important to ensure that the primary load is distributed to the quadriceps, glutes and hamstrings, rather than the lower back. This not only optimizes the effectiveness of the exercise, but also significantly reduces the risk of muscle imbalance and potential injury.

When analyzing your form in front squat, we will look at three things:

• The position of your elbows. They should be pointing up, not down. This helps to keep the spine straight.
• Knee alignment. The knees should be in line with the toes, not bent inwards.
• Width of feet. Not too wide or too narrow, they should be about shoulder width.

We will not see how to define these heuristics here, but you can look at the library’s source code. However, we will see an example of how to apply them:

import cv2
import numpy as np
import matplotlib.pyplot as plt

from ai_trainer.drawing import draw_pose, draw_text
from ar_trainer.feedback.front_squat import give_feedback

SAMPLE_PATH_1 = "./assets/right_posture.jpg"
SAMPLE_PATH_2 = "./assets/wrong_posture.jpg"

sample_1 = cv2.imread(SAMPLE_PATH_1)
sample_2 = cv2.imread(SAMPLE_PATH_2)
samples = [ cv2.cvtColor(sample_1, cv2.COLOR_BGR2RGB),
  cv2.cvtColor(sample_2, cv2.COLOR_BGR2RGB)]

f, axarr = plt.subplots(1,2)
for i, sample in enumerate(samples):
  img_h, img_w = sample.shape[:2]
  kps = blazepose_model.predict([sample])[0]

  probs = kps.T[3]
  if not any(probs<0):
    #denormalize keypoints:
    x, y, z = kps.T[:3]
    x_img = x * img_w
    y_img = y * img_h
    pose_3d = np.column_stack((x_img, y_img, z))
    pose_2d = np.column_stack((x_img, y_img))

    # Plot keypoints on image
    sample = draw_pose(
                image=sample,
                keypoints=pose_2d,
                disposition="mediapipe",
                thickness=2,
              )

    feedback, possible_corrections = give_feedback(pose_3d)
    number_corrections = len(list(feedback.keys())[1::])
    y_text_pos = 0
    for correction in possible corrections:
      y_text_pos+=25
      if correction in list(feedback.keys()):
        sample = draw_text(
                      image=sample,
                      text=feedback[correction],
                      origin=(10, y_text_pos),
                      font_scale=0.8,
                      color=(200,50,50),
                      thickness=2,
        )
  axarr[i].imshow(sample)
  axarr[i].axis("off")

Output:

Alberto performing a front squat. On the left, Alberto’s form is excellent. On the right, his elbows are pointing down. This has been highlighted by our system and Alberto will be able to correct this. — by the author

Step 4: Analyze your video

Now we will see how we can process a video of an athlete performing a front squat and get feedback on their form. You can find a sample video in “assets/left_side_cut.mp4" inside the repository.

import cv2
import numpy as np
from tqdm import tqdm

from ai_trainer.models import BlazePoseModel
from ai_trainer.drawing import draw_pose, draw_text
from ai_trainer.feedback.front_squat import give_feedback

VIDEO_PATH = "./assets/left_side_cut.mp4"
SAVE_VIDEO_PATH = VIDEO_PATH[:-4] + "_processed.mp4"

BLAZEPOSE_MODEL_PATH = "./models/blazepose_full.onnx"
blazepose_model = BlazePoseModel(model_path=BLAZEPOSE_MODEL_PATH)

# Read video:
cap = cv2.VideoCapture(VIDEO_PATH)
if cap.isOpened() is False:
    print("Error reading the video file")

# Get video configuration
nbr_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
frame_size = int(cap.get(3)), int(cap.get(4))  # frame_w, frame_h
img_w, img_h = frame_size
fps = int(cap.get(cv2.CAP_PROP_FPS))

# Create video writer
# Get video codec
if VIDEO_PATH[-4:] == ".mp4" or VIDEO_PATH[-4:] == ".MOV":
    fourcc = cv2.VideoWriter_fourcc(*"MP4V")
elif VIDEO_PATH[-4:] == ".avi":
    fourcc = cv2.VideoWriter_fourcc(*"XVID")

output = cv2.VideoWriter(SAVE_VIDEO_PATH,
                        fourcc,
                        fps, frame_size)

# Iterate through frames
print("💪 Evaluating your front squat technique...")
for i in tqdm(range(int(nbr_frames))):
    # Read the video file
    ret, frame = cap.read()

    # Check we have frames:
    if frame is None:
        break
    copy_frame = frame.copy()
    kps = blazepose_model.predict([
        cv2.cvtColor(copy_frame, cv2.COLOR_BGR2RGB)
        ])[0]  # batch size is 1

    probs = kps.T[3]
    if not any(probs<0):
        # denormalize keypoints:
        x, y, z = kps.T[:3]
        x_img = x * img_w
        y_img = y * img_h
        pose_3d = np.column_stack((x_img, y_img, z))
        pose_2d = np.column_stack((x_img, y_img))

        # plot keypoints on image
        copy_frame = draw_pose(
            image=copy_frame,
            keypoints=pose_2d,
            disposition="mediapipe", # blazepose keypoints are in mediapipe format
            thickness=2,
        )
        feedback, possible_corrections = give_feedback(pose_3d)

        y_text_pos = 0
        for correction in possible_corrections:
            y_text_pos+=25
            if correction in list(feedback.keys()):
                copy_frame = draw_text(
                    image=copy_frame,
                    text=feedback[correction],
                    origin=(10, y_text_pos),
                    font_scale=0.8,
                    color=(50, 50, 250),
                    thickness=2,
                )
    output.write(copy_frame)

cap.release()
output.release()

Output:

AI trainer

Ideas on how to improve the AI trainer

• Add more exercises. E.g. deadlift, bench press, snatch.
• Add automatic rep counting.
• Add exercise classification/identification.
• Add a web/mobile app to make it more user friendly.
• Enable real time (sound) alerts for real time form feedback.

One could even analyze the movement of the barbell more closely. For instance, in the front squat exercise, the barbell should follow an almost vertical movement. If we analyze the position of the barbell (approximated by the average position of the athlete’s wrists) in the previous video, we obtain a graph like this one:

Evolution of barbell’s position (X = side movement, Y = vertical movement, Z =frontal movement) in time. — by the author

From this graph alone we could extract the number of repetitions as well as feedback on the barbell movement.

Conclusions

Although AI is still a long way from replacing real trainers (in terms of expertise and motivatational skills), it has been shown that it can now make your workouts safer and more effective.

I hope the tutorial motivated you to develop your own and tailored AI trainer and, who knows, even go to the gym!

🔄 If you enjoyed the article, you can follow me on LinkedIN and Github🔄

Check out other articles from the team! :)

Revolutionizing Sports Advertisement with Generative AI.

Rebuild Your Home in 3D with Gaussian Splatting