Skip to yearly menu bar Skip to main content


Poster

Focal Plane Visual Feature Generation and Matching on a Pixel Processor Array

Hongyi Zhang · Laurie Bose · Jianing Chen · Piotr Dudek · Walterio Mayol-Cuevas


Abstract: Pixel Processor Arrays (PPAs) are vision sensors that embed data and processing into every pixel element. PPAs can execute visual processing directly at the point of light capture, and output only sparse, high-level information. This is in sharp contrast with the conventional visual pipeline, where whole images must be transferred from sensor to processor. This sparse data readout also provides several major benefits such as higher frame rate, lower energy consumption and lower bandwidth requirements. In this work, we demonstrate generation, matching and storage of binary descriptors for visual keypoint features, entirely upon PPA with no need to output images to external processing, making our approach inherently privacy-aware.Our method spreads descriptors across multiple pixel-processors, which allows for significantly larger descriptors than any prior pixel-processing works. These large descriptors can be used for a range of tasks such as place and object recognition. We demonstrate the accuracy of our in-pixel feature matching up to $ \sim$94.5%, at $\sim$210fps, across a range of datasets, with a greater than $100\times$ reduction in data transfer and bandwidth requirements over traditional cameras.

Live content is unavailable. Log in and register to view live content