On the Design of 2D Human Pose Estimation Networks using Accelerated Neuroevolution and Novel Keypoint Representations

Abstract

Motion capture is a very useful technology that is employed across many industries. Biomechanical analysis, film production, video game development, and virtual reality are among its diverse applications. However, traditional marker-based motion capture systems are limited by their invasiveness, excessive cost, and lack of portability. Human pose estimation represents a promising markerless alternative, where 3D human poses are estimated from RGB images obtained using single or multi-camera setups. The estimation of 2D poses serves as the main foundation for these systems. As such, the development of accurate and efficient 2D human pose estimation algorithms is critical to the overall advancement of markerless motion capture, and that is the focus of this thesis.

Two novel convolutional neural networks for 2D human pose estimation are presented, one for each of the two multi-person estimation paradigms (i.e., single-stage and two-stage). Motivated by the recent use of neural architecture search for convolutional neural network design, a novel neuroevolution framework is introduced and is leveraged in the design of a computationally efficient heatmap-based human pose estimation network for use in the two-stage paradigm. The neuroevolution was accelerated by a novel weight transfer scheme that relaxes the complete function-preservation constraint imposed by previous methods.

Recognizing the drawbacks of heatmaps, including the inherent issue of quantization error and the excessive computation required to generate and post-process large heatmap fields, two novel heatmap-free keypoint representations are introduced for modeling keypoint locations more efficiently. Drawing inspiration from single-stage object detectors, the representations are centered around modeling individual keypoints and sets of spatially related keypoints (i.e., poses) as objects. It is found that pose objects lend themselves well to single-stage human pose estimation, and a method is introduced that jointly learns human pose objects and keypoint objects and fuses the detections to exploit the strengths of both object representations.

At the time of development, both networks achieved state-of-the-art accuracy in their respective categories on the most established multi-person human pose estimation benchmarks when scaled. Moreover, they are more computationally efficient than previous networks as shown by having fewer parameters, fewer floating-point operations, and faster inference speed. The two-stage model is recommended for accuracy-critical scenarios with large computational budgets, whereas the single-stage model is more efficient and has greater potential for future research. It is hoped that these new models advance the current state of markerless motion capture systems based on human pose estimation.