Review: CPM — Convolutional Pose Machines (Human Pose Estimation)

Sik-Ho Tsang
May 13 · 5 min read
A sequential architecture that composed of convolutional networks for the prediction of right elbow

In this story, Convolutional Pose Machines (CPMs), by Carnegie Mellon University, is briefly reviewed.

  • CPM proposed a sequential architecture that composed of convolutional networks which directly operate on belief maps from previous stages, producing increasingly refined estimates for part locations.
  • CPM addresses the vanishing gradients problem during training by providing a natural learning objective function that enforces intermediate supervision.

This is a paper in 2016 CVPR with over 700 citations. (Sik-Ho Tsang @ Medium)


Outline

  1. Convolutional Pose Machines (CPM)
  2. Loss Function
  3. Evaluation

1. Convolutional Pose Machines (CPM)

  • (a): In the first stage, t = 1, with the input data xz as input:
  • where Yp is the pixel location of the p-th anatomical landmark (part), Z are the set of all locations in the image.
  • bp_1 is the the score predicted by the classifier g1 for assigning the p-th part at the first image.
  • At the output, there are P+1 belief maps, where P is the total number of parts, and +1 is the background.
  • (b): In the subsequent stages, t > 1, with the input data and the contextual information from the preceding classifier:
  • Therefore the part location can be refined by the later stages as below:
Part location can be refined by the later stages
  • (c): As within the first stage, there are five convolutional layers followed by two 1×1 convolutional layers which results in a fully convolutional manner.
  • (d): Input data goes through four convolutional layers before adding with the belief maps obtained in the first stage. And then three more convolutional layers followed by two 1×1 convolutional layers are processed.
  • (e): As for later stage, the effective receptive field is increased which help to improve the accuracy as shown below.
The effective receptive field is increased which help to improve the accuracy
  • The accuracy improves as the effective receptive field increases, and starts to saturate around 250 pixels.
  • In practice, to achieve certain precision, the input cropped images is normalized to size 368×368.
  • The receptive field of the first stage shown above is 160×160 pixels.
  • The second stage output on the belief maps of the first stage is set to 31×31, which is equivalently 400×400 pixels on the original image.

2. Loss Function

  • The cost function aimed to minimize at the output of each stage at each level is:
  • The summation of the costs of all stages is considered as the the costs of all stages:
  • By enforcing supervision in intermediate stages through the network, it can somehow address the vanishing gradient problem as the intermediate loss functions replenish the gradients at each stage.
  • In early epochs, as we move from the output layer to the input layer, we observe on the model without intermediate supervision, the gradient distribution is tightly peaked around zero because of vanishing gradients.
  • The model with intermediate supervision has a much larger variance across all layers, suggesting that learning is indeed occurring in all the layers thanks to intermediate supervision.

3. Evaluation

3.1. Ablation Study

Different Training Methods (Left), Number of Stages (Right)
  • PCK is measured which is the error tolerance is normalized with respect to the part target.
  • Left: With end-to-end training, it is better than stagewise training, and stages without intermediate supervision.
  • Right: The performance increases monotonically until 5 stages, diminishing returns at the 6th stage.

3.2. MPII Human Pose Dataset

3.3. Leeds Sports Pose (LSP) Dataset

  • 11000 training images and 1000 testing images.
  • As shown above, CPM outperforms Tompson NIPS’14.

3.4. FLIC Dataset

FLIC Dataset

Sik-Ho Tsang

Written by

PhD, Researcher. I share what I've learnt and done. :) My LinkedIn: https://www.linkedin.com/in/sh-tsang/

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade