AI study creates faster and more reliable software

University of Stirling researchers have trained ChatGPT to produce faster versions of a software program. Making software run faster and more reliably is challenging and time-consuming for software developers.

The study led by Dr. Sandy Brownlee used ChatGPT—an artificial intelligence (AI) large language model (LLM)—to automatically update software by asking it to make improvements to computer coding. The results of the research could be used to improve mobile apps to make them more responsive and less draining on smartphone batteries.

The paper, "Enhancing Genetic Improvement Mutations Using Large Language Models," was presented by Dr. Brownlee on December 8 at the Symposium on Search-Based Software Engineering 2023, in San Francisco, part of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering.

Dr. Brownlee, a senior lecturer in the University of Stirling's Computing Science and Mathematics Division in the Faculty of Natural Sciences, said, "We found that, on the open source project we used as a case study, a LLM was able to produce faster versions of the program around 15% of the time, which is half as good again as the previous approach.

"Most interestingly was that the LLM was able to take examples from other parts of the program that we hadn't explicitly told it about in order to make these improvements.

"The most tangible benefit is in your pocket—mobile apps that run more efficiently mean that your battery lasts longer, and the apps will be more responsive when in use."

More efficient improvements

Software is becoming increasingly complex and difficult to maintain. It is also having a growing environmental impact as computers consume more and more energy, so creating more efficient automatic improvements to software are crucial if countries are to achieve net-zero commitments.

Dr. Brownlee said, "There is a trade-off here because LLMs cost a lot of energy to make and use but if they can be used to improve a piece of software that is run many times, the payoff may be worth it."

Dr. Brownlee hopes that the research could help software developers who are striving to create more efficient programs.

He said, "The nature of software developers' roles will change if automated improvement to software becomes commonplace, moving to a higher level of design, though that is continuing the direction of travel we've seen for decades.

"It's quite possible to be concerned about all this—AI writing computer code sounds like the beginning of a sci-fi disaster plot—but this is not going to lead to an AI takeover.

"We're making tools that support developers by providing more information and it's still critical to have a human involved to use that information effectively."