PHD: Personalized 3D Human Body Fitting with Point Diffusion

We introduce PHD, a novel approach for 3D human pose and shape estimation that leverages user identity information from videos to improve pose estimation accuracy and shape consistency. Unlike traditional methods designed to be user-agnostic and optimized for generalization, our pipeline precomputes the body shape and then employs a personalized pose fitting process conditioned on the body shape and input image. We observe that while existing methods commonly improve 2D alignment by refining the pose with constraints derived from the 2D image, the lack of 3D pose prior often reduces pose plausibility, thereby compromising 3D accuracy. To address this, we integrate a body shape-conditioned 3D pose prior, implemented as a Point Diffusion model, to iteratively guide pose fitting via a Point Distillation loss. Our results demonstrate that our 3D pose prior significantly prevents artifacts introduced by 2D-only constraints, which consequently improves the pose accuracy. In addition, our 3D prior-driven fitting method is highly versatile and can be seamlessly combined with state-of-the-art 3D pose estimators to improve pose accuracy.