New research works to improve image classification and analysis

A new field promises to usher in a new era of using machine learning and computer vision to tackle small and large-scale questions about the biology of organisms around the globe. The field of imageomics aims to help explore fundamental questions about biological processes on Earth by combining images of living organisms with computer-enabled analysis and discovery.

Wei-Lun Chao, an investigator at The Ohio State University's Imageomics Institute and a distinguished assistant professor of engineering inclusive excellence in computer science and engineering at Ohio State, gave an in-depth presentation about the latest research advances in the field last month at the annual meeting of the American Association for the Advancement of Science.

Chao's AAAS presentation, titled "An Imageomics Perspective of Machine Learning and Computer Vision: Micro to Global," was part of the session "Imageomics: Powering Machine Learning for Understanding Biological Traits."

Chao and two other presenters described how imageomics could transform society's understanding of the biological and ecological world by turning research questions into computable problems. Chao's presentation focused on imageomics' potential application for micro to macro-level problems.

"Nowadays we have many rapid advances in machine learning and computer vision techniques," said Chao. "If we use them appropriately, they could really help scientists solve critical but laborious problems."

While some research problems might take years or decades to solve manually, imageomics researchers suggest that with the aid of machine and computer vision techniques—such as pattern recognition and multi-modal alignment—the rate and efficiency of next-generation scientific discoveries could be expanded exponentially.

"If we can incorporate the biological knowledge that people have collected over decades and centuries into machine learning techniques, we can help improve their capabilities in terms of interpretability and scientific discovery," said Chao.

One of the ways Chao and his colleagues are working toward this goal is by creating foundation models in imageomics that will leverage data from all kinds of sources to enable various tasks. Another way is to develop machine learning models capable of identifying and even discovering traits to make it easier for computers to recognize and classify objects in images, which is what Chao's team did.

"Traditional methods for image classification with trait detection require a huge amount of human annotation, but our method doesn't," said Chao. "We were inspired to develop our algorithm through how biologists and ecologists look for traits to differentiate various species of biological organisms."

Conventional machine learning-based image classifiers have achieved a great level of accuracy by analyzing an image as a whole, and then labeling it a certain object category. However, Chao's team takes a more proactive approach: Their method teaches the algorithm to actively look for traits like colors and patterns in any image that are specific to an object's class—such as its animal species—while it's being analyzed.

This way, imageomics can offer biologists a much more detailed account of what is and isn't revealed in the image, paving the way to quicker and more accurate visual analysis. Most excitingly, Chao said, it was shown to be able to handle recognition tasks for very challenging fine-grained species to identify, like butterfly mimicries, whose appearance is characterized by fine detail and variety in their wing patterns and coloring.

The ease with which the algorithm can be used could potentially also allow imageomics to be integrated into a variety of other diverse purposes, ranging from climate to material science research, he said.

Chao said that one of the most challenging parts of fostering imageomics research is integrating different parts of scientific culture to collect enough data and form novel scientific hypotheses from them.

It's one of the reasons why collaboration between different types of scientists and disciplines is such an integral part of the field, he said. Imageomics research will continue to evolve, but for now, Chao is enthusiastic about its potential to allow for the natural world to be seen and understood in brand-new, interdisciplinary ways.

"What we really want is for AI to have strong integration with scientific knowledge, and I would say imageomics is a great starting point towards that," he said.