Abstract : Real-time anomaly and violent action detection from low-altitude aerial video feeds presents many computational challenges as a result of perspective distortion, camera motion, and the limited processing power available on-board. Frame-level 2D convolutional neural networks (CNNs) have a high false positive rate as a direct consequence of their inability to model the temporal dependencies that exist between consecutive frames of video. In an attempt to reduce these limitations, this research evaluates a decoupled, multi-stage spatiotemporal deep learning pipeline that was developed specifically to be executed on localised edge computing platforms. The initial step in the pipeline involves localizing humans in each frame of video using an ultra-lightweight YOLOv8-Nano detector; the relationship between humans in different frames is established through a multi-object tracking (MOT) ByteTrack algorithm. After establishing relationships between humans across multiple frames of video, action dynamics are classified using a spatiotemporal ResNet-3D (R3D-18) network which uses a sliding temporal window of frames. Evaluation of the proposed pipeline demonstrated that it achieved 91.30% classification accuracy and an F1-score of 0.91 when evaluated against real-world aerial surveillance and battle datasets. Additionally, prototype testing demonstrated an approximate processing throughput of 20 frames per second, while the spatiotemporal model achieved a 10.80 percentage-point improvement in classification accuracy compared with the frame-level 2D CNN baseline.

Keyword : Action Recognition, ByteTrack, Computer Vision, Edge Intelligence, Spatiotemporal Modeling, UAV Surveillance.