Simulating Dual-Pixel Images From Ray Tracing For Depth Estimation

Many studies utilize dual-pixel (DP) sensor phase characteristicsfor various applications, such as depth estimation and deblurring.However, since the DP image features are entirely determined by the camera hardware, DP-depth paired datasets are very scarce, especially when performing depth estimation on customized cameras.To overcome this, studies simulate DP images using ideal optical system models.However, these simulations often violate real optical propagation laws,leading to poor generalization to real DP data.To address this, we investigate the domain gap between simulated and real DP data, and propose solutions using the Simulating DP images from ray tracing (Sdirt) scheme.The Sdirt scheme generates realistic DP images via ray tracingand integrates them into the depth estimation training pipeline.Experimental results show that models trained with Sdirt-simulated imagesgeneralize better to real DP data.The code and simulated datasets will be available on GitHub.