Idea

Most of the previous works on human pose estimation are based on the two-layer part-based model. The first layer focuses on local (body) part appearance and the second layer imposes the contextual relations between local parts. These pose estimation methods using part-based models are usually sensitive to noise and the graphical model lacks expressiveness to model complex human poses. Furthermore, most of these methods search for each local part independently and the local appearance may not be sufficiently discriminative for identifying each local part reliably.

In this work, we propose a dual-source CNN (DS-CNN) based method for human pose estimation without using any explicit graphical model. This proposed method utilizes both the local part appearance in image patches (part patch) and the holistic view of the full human body (body patch) for more accurate human pose estimation. By adding a binary mask to the body patch, we novelly integrate global and local information to one input (body patch). By taking both the local patch and the body patch as inputs, the proposed DS-CNN performs a unified learning to achieve both joint detection, which determines whether an object proposal contains a body joint, and joint localization, which finds the exact location of the joint in the object proposal.