HAMLET: A platform to simplify AI research and development

Machine learning (ML) algorithms have proved to be highly valuable computational tools for tackling a variety of real-world problems, including image, audio and text classification tasks. Computer scientists worldwide are developing more of these algorithms every day; thus, keeping track of them and quickly finding or accessing those introduced in the past is becoming increasingly challenging.

With this in mind, researchers at Purdue University and University of Cincinnati recently created HAMLET, a platform that could help computer scientists and developers to browse through existing machine learning models and train or evaluate their own algorithms, thus aiding their research and development efforts. This platform, presented in a paper pre-published on arXiv, could ultimately democratize machine learning models developed around the world, allowing research teams to share their models with each other.

"Organizing and keeping track of the machine learning algorithms and datasets has always been a major challenge for us, as well for as many other researchers in the field," Ahmad Esmaeili, one of the researchers who carried out the study, told TechXplore. "This becomes even more critical when the number of ML solutions and components continues to grow over time and from one project to another. When developing HAMLET, we have strived to create a platform that meets the needs above by not only administering the available ML contributions and assets in a distributed way, but also facilitating actions such as accessing, comparing and evaluating those resources effectively."

HAMLET, which stands for Hierarchical Agent-based Machine LEarning plaTform, is composed of a group of AI agents trained to "manage" a large group of ML algorithms, related resources (e.g., datasets) and tasks that ML models are trained to complete. The researchers defined the skills of the artificial agents that "manage" the platform, which are arranged at different levels of a hierarchy based on the algorithms, data or task that they represent.

"The HAMLET platform starts with an empty structure and continues to autonomously grow with the introduction of new ML resources/queries over time," Esmaeili explained. "Being based on multi-agent systems, HAMLET can be distributed over a network of computers and devices; thus, there is no limitation on the size and type of algorithms/data that it can host."

The HAMLET platform has a user-friendly interface and flexible query structure. Researchers can use it to perform a variety of tasks, for instance to train and test their algorithms, both individually and in batches.

To test its effectiveness, Esmaeili and his colleagues used it to complete 120 training and four batch testing tasks on a simulated environment developed with SPADE (Smart Python Agent Development Environment). They repeatedly tested and trained 24 ML algorithms using nine renowned datasets for training AI agents. The results of their experiments suggest that HAMLET is a highly promising and useful tool for training and testing ML algorithms.

"There is no doubt that machine learning approaches are becoming increasingly prevalent," Esmaeili said. "HAMLET facilitates the democratization of ML solutions and helps the ML research communities, regardless of their geographical locations, easily share and keep track of their methods and resources."

In the future, the platform created by Esmaeili and his colleagues could be used by researchers worldwide to train new ML algorithms on multiple datasets, identify existing models for specific purposes or evaluate new algorithms and compare their performance to that of other existing ones. On HAMLET, all of these tasks can easily be completed via a single query.

"This project is in its infancy and can be ameliorated in many aspects to ensure that it better meets the current research and industrial needs," Esmaeili said. "In our next studies, we plan to continue working on supporting more sophisticated algorithms, the survivability of the platform against failures, merging multiple platforms, and the privacy of accessing data/algorithms."

© 2020 Science X Network