Scientists develop mobile system for object detection, image analysis in disaster response

A team of researchers from the Department of Energy's Oak Ridge National Laboratory has created a prototype system for detecting and geolocating damaged utility poles in the aftermath of natural disasters such as hurricanes.

The system, which is detailed in the journal Photogrammetric Engineering and Remote Sensing, is designed to run on edge computing hardware mounted on a quadcopter or other uncrewed aerial vehicle, allowing it to function when local infrastructure is damaged or destroyed.

The team from ORNL's Geospatial Science and Human Security Division used machine learning algorithms and onboard imaging hardware to accurately detect and assess damage to utility poles while uploading location information to a central processing hub, called the Environment for Analysis of Geo-Located Energy Information, or EAGLE-I. This information can be relayed to utility companies, first responders or other groups supporting energy infrastructure.

The edge computing platform is one of several projects designed for incorporation into the EAGLE-I system, a multifaceted real-time situational awareness tool for the nation's energy infrastructure. EAGLE-I allows its users to monitor energy infrastructure assets, report energy outages, display potential threats to energy infrastructure and coordinate emergency response and recovery.

Among the edge computing team's key concerns are efficiency and practicality.

"One of the main drivers of our work is to make a system that can be afforded and run by local and state governments," said ORNL's David Hughes, the project's principal investigator and an expert in airborne and satellite-based image processing and analysis. "So we work with affordable sensors and platforms."

While the affordable hardware does have limited image resolution and rate capture, Hughes and his team have worked hard to optimize their machine learning analysis software to ensure these limitations are manageable.

In addition to navigating concerns about hardware and affordability, the group is working to ensure the onboard detection and identification system can accurately identify utility poles and their status in a variety of situations.

"One of our biggest challenges right now is just getting sufficient training data," said ORNL's Jordan Bowman, the project's engineering specialist in machine learning. "Deep learning projects often place a lot of emphasis on collecting very large quantities of imagery, but we're a bit more limited in the total number of photos we can collect and annotate."

To address this issue and acquire more usable training data, the group has sent teams to collect images of damage following extreme weather events (such as Hurricane Ian) and has partnered with local energy companies to collect even more training data from which to build a robust detection and analysis system.

"This pole detection project is just our first step into 'AI on the edge,'" Hughes said. "Our intent is to expand into multiple observables—substations, for example—and be able to classify them as damaged or undamaged infrastructure."

When discussing AI on the edge, Hughes is not only referring to the cutting-edge methods he and his team are using to design their image analysis tools or the UAS platform upon which they will run. He's also talking about a broader class of new AI computing projects in which AI applications are deployed in devices close to users rather than in a cloud computing facility or private data center. These so-called edge computing projects allow for improved security and efficiency, as well as increased uptime and decreased costs in many cases.

AI on the edge can be useful in a broad range of applications, and the interdisciplinary ORNL team is already considering several new research avenues enabled by their edge computing project.

"The image analytics capabilities of the smaller, more affordable sensors that we're making allow for a lot of things that were previously impossible because of price and resolution limitations," said Lexie Yang, an ORNL researcher and expert in computer vision and high-performance machine learning.

Within the project, Yang works to integrate the machine learning and computer vision components with the rest of the onboard system. "We're looking to expand to more observables and more types of disasters. For example, wildfire damage to energy infrastructure, flooding and so forth," she said.

Hughes added, "We are also starting relationships with government organizations that do search and rescue, where this work will be really helpful."

The team's new edge computing system will improve damage assessment and resource allocation in disaster response and promises a new generation of remote sensing technology for improved preparedness and response to a wide range of threats to national and human security.