TinyLPFNet: A Near-Imager FPGA Accelerator for Real-Time Human Activity Recognition

Camera-based human activity recognition (HAR) for always-on edge deployment is constrained not only by arithmetic cost but also by persistent data transfer to off-chip memory, which incurs substantial delay, memory traffic, and excessive storage demand. This brief presents TinyLPFNet, an event-triggered near-imager FPGA accelerator that confines inference, buffering, and control entirely within a single reconfigurable device, eliminating external memory access from the deployed pipeline. This brief proposes three techniques to improve HAR efficiency: (i) a frame-differencing event-detection scheme that activates inference only when meaningful scene changes are detected, thereby suppressing redundant frame transmission; (ii) a spatial activity mask that exploits scene sparsity to avoid unnecessary computation in inactive regions by reducing data movement and disabling idle processing elements (PEs); and (iii) structured Legendre basis functions that parameterize the depthwise kernels during training, with the resulting kernels materialized as standard 3 × 3 weights before INT8 deployment, preserving a regular DSP-compatible MAC structure without Legendre-specific inference hardware. Evaluated on a seven-class HAR dataset, the deployed system achieves 97.59% accuracy, 0.66 ms end-to-end latency, and real-time throughput at 125MHz on a Kintex-7 XC7K325T platform.