This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:


peer-reviewed publication

trusted source


Eye-tracking during building inspections provides insight on how experts think

Eye-tracking during building inspections provides insight on how experts think
Mariantonieta Gutierrez Soto, assistant professor of engineering design in the Penn State School of Engineering Design and Innovation, uses Tobii eye-tracking glasses to assess damages in row buildings in Mayfield, Kentucky after the region endured tornadoes in December 2021. Credit: Mariantonieta Gutierrez Soto

After a building failure due to natural disasters or poor structural design, safety inspectors must enter a structure to assess the damage before occupants can return. Researchers in the Penn State Department of Architectural Engineering have studied how building inspectors make their safety assessments, by analyzing their gaze patterns with eye-tracking software. Eventually, the eye-tracking data could be used to code autonomous robots, like drones, to conduct building assessments in place of humans.

The researchers' results were published in Scientific Reports.

"We are looking for a way to capture how an inspector thinks and makes assessments on a to understand their intentions and diagnostic choices," said corresponding author Rebecca Napolitano, assistant professor of architectural engineering. "Studying where they look—and for how long—at certain points on a building can help us do that."

Researchers tasked 10 architectural engineering graduate students to assess two building facades while wearing Tobii eye-tracking glasses. The glasses have two cameras to measure movements and positioning simultaneously. One camera faces the object, and the other is trained on the user's eyes to measure pupil dilation and gaze direction.

Study participants were given unlimited time to assess two sections of the facades of two steel-framed buildings in central Pennsylvania. Researchers chose the sites due to the buildings' various observable issues, including plant growth, peeling paint, degrading bricks, cracks, water cooling and intrusion, and erosion at the foundation.

Participants—who all had some building diagnostics experience and received the same training ahead of the experiment on building inspection—spent varying amounts of time gazing at different points of the facades. Some missed more obvious issues and focused for long periods on smaller issues, such as cracks.

"Some of the issues were more or less obvious, and eventually everyone found most of the same things, but their approaches to the exercise were very different," Napolitano said. "We learned that people, depending on their backgrounds, have biases when looking at structures."

Eye-tracking during building inspections provides insight on how experts think
Architectural engineering researchers collected eye-tracking data from 10 participants, who assessed two building facades while wearing Tobii eye-tracking glasses. The figure shows a comparison among the heat maps and gaze plots of participants. Credit: Rebecca Napolitano

Someone accustomed to looking for certain issues, such as water damage, for example, will spend more time on that particular issue, Napolitano said.

The results differed from studies in other fields that have conducted eye-tracking assessments, such as in medical imaging research and psychology, according to the researchers. Experts in those fields were shown to spend the same amount of time on each point of a page, including when encountering abnormalities. The building inspection participants, on the other hand, tended to focus for longer periods on observable issues.

The broad differences in how people assess the buildings present a challenge for coding drones to perform the same inspections, the researchers said.

"It would be hard to take an average of all the data since the study participants had such different results," Napolitano said. "But it is one small step in understanding how inspectors think, with the long-term goal of creating an algorithm to inform a drone."

In the future, the researchers plan to carry out another eye-tracking experiment, this time using a 3D stimulus, such as a room inside of a building, as opposed to a building facade, which is considered 2D. Toward that goal, some members of Napolitano's lab collected sample 3D data inside damaged structures during a visit to Mayfield, Kentucky, after the area's December 2021 tornado, which the group intends to analyze and code.

"We plan to use the collected field data to create synthetic data, which we can then use to learn more about human-robot relationships," Napolitano said.

In addition to Napolitano, the co-authors included Muhammad Rakeh Saleem, a Penn State doctoral student in architectural engineering, and Robert Mayne, a math instructor from Chariho Regional High School, Rhode Island, who assisted with the statistical analysis.

More information: Muhammad Rakeh Saleem et al, Analysis of gaze patterns during facade inspection to understand inspector sense-making processes, Scientific Reports (2023). DOI: 10.1038/s41598-023-29950-w

Journal information: Scientific Reports
Citation: Eye-tracking during building inspections provides insight on how experts think (2023, March 28) retrieved 19 May 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Engineers assess structural damage and resilience after Kentucky tornado


Feedback to editors