Simulating learning methodology: An approach to machine learning automation

As a fundamental technology of artificial intelligence, existing machine learning (ML) methods often rely on extensive human intervention and manually presetting, like manually collecting, selecting, and annotating data, manually constructing the fundamental architecture of deep neural networks, and determining the algorithm types and their hyperparameters of the optimization algorithms, etc. These limitations hamper the ability of ML to effectively deal with complex data and varying multi-tasks environments in the real world.

To address these existing challenges in current ML, the research team from Xian Jiaotong University has developed a novel approach called simulating learning methodology (SLeM). The core concept of SLeM is to simulate and extract the learning methodology of ML traditionally set by humans, transforming it into an automated learning process. Essentially, the SLeM framework represents an ML for ML paradigm, where ML tools are employed to design and optimize MLs fundamental components.

The team has developed a series of algorithms for ML automation based on the SLeM framework, demonstrating its effectiveness in enhancing the adaptive learning capabilities of existing ML methods.

"Recently, many AutoML methods have been proposed to achieve ML automation. However, most existing AutoML methods are heuristic in nature, making it difficult to establish a solid theoretical foundation. In contrast, the SLeM framework offers a unified mathematical formulation for ML automation and provides theoretical insights into the task transfer generalization capabilities of SLeM," says Prof. Zongben Xu, the lead author of this paper and an academician of the Chinese Academy of Sciences.

The development of advanced large language models (LLMs) has become a cornerstone of artificial intelligence, significantly expanding the capacity to solve a wide range of applications and tasks. However, the underlying theoretical evidence for LLMs' remarkable task generalization abilities has not been fully addressed by the ML community. The novel SLeM approach provides a promising perspective and tool for advancing the study and understanding of task generalization capabilities in large language models (LLMs).