Adaptive anomaly detection in traffic surveillance videos

Adaptive anomaly detection in traffic surveillance videos
Complete Adaptive Anomaly Detection Flow. Credit: Farhadi et al.

Researchers at Arizona State University (ASU) have recently developed a new method of detecting unusual activity in traffic surveillance videos, which is based on an adaptive anomaly detection (AAD) framework. Their method, outlined in a paper pre-published on arXiv, gathered promising results, performing better than techniques presented in previous studies.

"Due to an increase in the number of surveillance cameras in the world, the amount of data to be processed grows, along with the demand for manpower to interpret this data," Mohammad Farhadi Bajestani, one of the researchers who carried out the study, told Tech Xplore. "This motivates researchers to design systems that can detect anomalies inside a video, helping operators to analyze videos more efficiently."

Currently, most researchers developing surveillance systems use pre-defined facts to detect anomalies within video footage. In other words, their systems are trained on video datasets, gradually learning to detect anomalies in future videos.

"However, for human beings, the definition of anomaly can change over time," Farhadi explained. "For instance, when you are driving very fast, slow traffic is an anomaly and might catch your attention, yet when you are driving in traffic, a car running at high speed can raise your attention. This shows that anomaly is related to time and can vary according to the situation. In our research, we tried to create an adaptive anomaly system which can learn normal states in short periods of time and spot anomalies within different scenarios."

Adaptive anomaly detection in traffic surveillance videos
Pixel movement across the frame after ∆t. Credit: Farhadi et al.

The method developed by Farhadi and his colleagues is designed to detect normal situations in different parts of a video frame, identifying anomalies in the presence of unusual objects or movements. First, the system uses object recognition to identify objects and their location within a video frame. Once these objects are identified, it uses optical flow to identify changing flows of traffic in different parts of the frame. Finally, this information is used to detect anomalous movements or objects.

"We achieved this adaptive anomaly detection by creating two maps," Farhadi explained. "The first map outlines the average movement in frame parts, while the second records the probability of appearance of each object in different parts of frame. This two are always changing as a new frame comes in. For instance, at the center of the frame, we mostly see cars with specific movements, so if a bike at low speed appears, the system will consider it as an anomaly."

The system developed by the researchers somewhat resembles mechanisms. Just as human attention helps humans to identify anomalies without analyzing all data perceived, their method can spot temporal anomalies and focus on these, providing deeper analyses of what is happening at the time.

Adaptive anomaly detection in traffic surveillance videos
Optical Flow Outputs. Credit: Farhadi et al.

"Our system could be used inside , detecting anomalies on the road and either raising the operator's attention or running more complex AI analyses on the region in which the anomaly was detected," Farhadi said. "This could prevent car crashes, such as the one that took place in Arizona, in which one of Uber's autonomous cars killed a woman because the system failed to detect her on street. In a similar instance, our system might detect abnormality in incoming frames and raise the operator's attention, perhaps saving a life."

The researchers evaluated the performance of their adaptive detection (AAD) system comparing it to that of other baseline methods. They found that although it did not achieve state-of-the-art results, it performed better than previously developed techniques, with substantially faster running times.

"Having a robust and fast object detection system can be very useful for analyzing surveillance videos," Farhadi said. "Due to weaknesses in our system's object detection capabilities, we are now working on improving object detection in surveillance. We are also developing a new framework for running neural networks using FPGA, called CNNIOT, which will make it possible to run huge systems on small devices such as mobile devices or small robots."

More information: AAD: Adaptive Anomaly Detection through traffic surveillance videos. arXiv:1808.10044 [cs.CV].