Super Mario hackers' tricks could protect software from bugs, study finds

Video gamers who exploit glitches in games can help experts better understand buggy software, students at the University of Bristol suggest.

Known as "speedrunners," these types of gamers can complete games quickly by working out their malfunctions.

The students examined four classic Super Mario games, and analyzed 237 known glitches within them, classifying a variety of weaknesses. This research, presented at the Workshop on Games and Software Engineering (GAS) at the International Conference on Software Engineering (ICSE 2024), explores whether these are the same as the bugs exploited in more conventional software.

Nintendo's Super Mario is the quintessential video game. To understand the sorts of glitches speedrunners exploit, they examined four of the earliest Mario platforming games—Super Mario Bros (1985), Super Mario Bros. 3 (1988), Super Mario World (1990) and Super Mario 64 (1996).

While these games are old, they are still competitively run by speedrunners with new records reported in the news. The games are also well understood, having been studied by speedrunners for decades, ensuring that there are large numbers of well-researched glitches for analysis.

Currently, the world record time for conquering Super Mario World stands at a blistering 41 seconds. The team set out to understand 237 known glitches within them, classifying a variety of weaknesses to see if they can help software engineers make applications more robust.

In the Super Mario platforming games, Mario must rescue Princess Peach by jumping through an obstacle course of various platforms to reach a goal, avoiding baddies or defeating them by jumping on their heads. Players can collect power-ups along the way to unlock special abilities, and coins to increase their score. The Mario series of games is one of Nintendo's flagship products, and one of the most influential video game series of all time.

Dr. Joseph Hallett from Bristol's School of Computer Science explained, "Many early video games, such as the Super Mario games we have examined, were written for consoles that differ from the more uniform PC-like hardware of modern gaming systems.

"Constraints stemming from the hardware, such as limited memory and buses, meant that aggressive optimization and tricks were required to make games run well.

"Many of these techniques (for example, the NES's memory mapping) are niche and can lead to bugs, by being so different to how many programmers usually expect things to work.

"Programming for these systems is closer to embedded development than most modern software, as it requires working around the limits of the hardware to create games. Despite the challenges of programming these systems, new games are still released and retro-inspired."

Categorizing bugs in software allows developers to understand similar problems and bugs.

The Common Weakness Enumeration (CWE) is a category system for hardware and software weaknesses and vulnerabilities. The team identified seven new categories of weakness previously unspecified.

Dr. Hallett said, "We found that some of the glitches speed runners use don't have neat categorizations in existing software defect taxonomies and that there may be new kinds of bugs to look for in more general software."

The team thematically analyzed with a code book of existing software weaknesses (CWE)—a qualitative research method to help categorize complex phenomena.

He continued, "The cool bit of this research is that academia is starting to treat and appreciate the work speedrunners do and study something that hasn't really been treated seriously before.

"By studying speedrunners' glitches we can better understand how they do it and whether the bugs they use are the same ones other software gets hacked with.

"It turns out the speedrunners have some tricks that we didn't know about before."

Now the team have turned their hand to studying Pokémon video games.