MeshMamba: State Space Models for Articulated 3D Mesh Generation and Reconstruction

In this paper, we introduce MeshMamba, a neural network model for learning 3D articulated mesh models by employing the recently proposed Mamba State Space Models (SSMs). MeshMamba is efficient and scalable in handling a large number of input tokens, enabling the generation and reconstruction of body mesh models with approximately 10,000 vertices. The key to effectively learning MeshMamba is the serialization technique of mesh vertices into the orderings that are easily processed by Mamba. This is achieved by sorting the vertices based on the body part annotations or the 3D vertex locations of a template mesh, such that the ordering respects the structure of articulated shapes. Based on MeshMamba we design 1) MambaDiff3D, a denoising diffusion model for generating 3D articulated meshes, and 2) Mamba-HMR, a 3D human mesh recovery model which reconstructs a human body shape pose from a single image. Experimental results showed that MambaDiff3D can generate dense 3D human meshes in clothes, with grasping hands etc. and outperforms previous approaches in the 3D human shape generation task. Also, Mamba-HMR extends the ability of previous non-parametric human mesh recovery approaches, which were limited in handling body-only poses using around 500 vertex tokens, to the whole-body setting with face and hands, while achieving competitive performance in (near) real-time.