Tracking human motion using wearable inertial measurement units (IMUs) overcomes occlusion and environmental limitations inherent in vision-based approaches.However, such sparse IMU tracking also compromises translation estimates and accurate relative positioning between multiple individuals, as inertial cues are inherently self-referential and provide no direct spatial reference or relational information about others.In this paper, we present a novel approach that leverages the distances between the IMU sensors worn by one person as well as between those across multiple people.Our method Inter Inertial Poser derives these absolute inter-sensor distances from ultra-wideband ranging (UWB) and inputs them into structured state-space models to integrate temporal motion patterns for precise 3D pose estimation.Our novel coarse-to-fine optimization process further leverages these inter-sensor distances for accurately estimating the trajectories between individuals. To evaluate our method, we introduce Inter-UWB, the first IMU+UWB dataset for two-person tracking, which comprises 200\,minutes of motion recordings from 14\,participants. Our results show that Inter Inertial Poser outperforms the state-of-the-art methods in both accuracy and robustness across synthetic and real-world captures, demonstrating the promise of IMU+UWB-based multi-human motion capture in the wild.