Using AI, cars can detect potholes in real time

The Korea Institute of Civil Engineering and Building Technology (KICT) has announced the development of an 'AI-based automatic pothole detection system'. The system is designed to be installed on the windshield of a vehicle to detect potholes on the road surface in real-time. Potholes can damage cars and may even lead to life-threatening accidents.

In particular, potholes can cause problems in the rainy seasons. In Seoul, when the record was set for heavy rain in August 2020, more than 7,000 pothole reports were received in August 2020 when a record downpour hit the city. In Korea, the number of potholes reported across country from 2016 to 2018 was 657,993. Total damage compensation amounted to 4.6 billion won nationwide, and road repair costed 1.7 trillion won. If a vehicle passes over a pothole at high speed without noticing it, the vehicle may break away from the driving lane, hence a threat to the driver's life.

Road surface management starts with quickly detecting damaged sections, and this involves vibration-, laser scanning-, and image recognition-based detection technologies. In particular, with the recent development of the detection technology using deep neural networks, image recognition-based road surface management methods are receiving attention. In addition, the image-based technique can be used with personal devices, such as a vehicle or smartphone camera. This makes it easier for local governments, which have relied on visual inspections by humans, to employ the technology.

A research team at KICT led by Dr. Seungki Ryu developed this system that detects potholes in real-time by photographing the road surface while driving with a vision sensor installed on the windshield of a vehicle. The AI inference model semantically segments damages on the road surface using an encoder-decoder network based on the FCN (fully convolutional neural network) architecture.

A common problem in image-based detection is that even at the same location images can vary in the pixel unit information depending on changes in the external environment. In particular, it may be challenging to identify damages on the road surface with the AI inference model as the brightness of the road surface changes over time. To solve this issue, a new CNN (convolutional neural network) model for image preprocessing was developed and combined with the semantic segmentation model to show robust detection performance with road images taken under different brightness conditions.

This technology consists of a mobile app for gathering data using the AI model and a map-based cloud server platform to identify potholes based on data transmitted from the mobile app. Currently, several local governments in Korea, such as Gwangju Metropolitan City, Goyang-si, and Gimhae-si are piloting this technology. Dr. Ryu's research team aims to further expand and introduce this technology to other local governments.

Dr. Ryu said, "It is essential to maintain road facilities in good condition in the coming era of autonomous vehicles. This AI-based technology will make effective road surface management much easier."