ML-Enhanced Data Reduction for Autonomous Systems

In the embedded electronics of autonomous systems, the proliferation of frontend sensors has led to a surge in raw data generation, imposing substantial power consumption and bandwidth requirements. In an effort to suppress the sheer volume of generated data, we explore intelligent data reduction techniques such as filtering or compression for autonomous systems.

Previously we have demonstrated a near-pixel temporal frame filtering mechanism for autonomous driving, enabled by a software-hardware co-design approach. The proposed near-pixel AI filters out redundant image frames early in the imaging pipeline. This approach not only helps to lower the power consumption of the backend processors due to essentially reduced frame rate, but also guarantees that a variety of downstream tasks can effectively operate on the transmitted raw frames.

To support efficient execution of the frame filtering network, we also co-design a near-pixel digital CIM accelerator with the novel features of using NOR gate arrays as both multipliers and buffers between network layers, support for frontend demosaicing, and sparsity-aware adder tree.