March 18, 2019 feature
A new method for understanding ancient coin images
Two researchers at the University of St. Andrews, in Scotland, have recently developed a new machine learning-based method for understanding images of ancient coins. Their study, pre-published on arXiv applies computer vision and machine learning to ancient numismatics.
"My research in this field was a product of bringing two passions together: my ongoing interest in ancient coins (I have a large collection myself) and the state of the art in AI," Ognjen Arandjelovic, one of the researchers who carried out the study, told TechXplore. "In 2010, I wrote a paper on the topic and to my surprise, as this is typically a niche interest, it attracted a lot of attention."
Most previous studies have tried to better understand ancient coins using generic object recognition techniques. Arandjelovic's knowledge and understanding of ancient numismatics, however, led him to believe that these approaches are far from optimal and encouraged him to develop alternative methods. Over the past decade or so, he has published a series of papers that deviate from the aforementioned pattern.
Most existing state-of-the-art methods for the analysis of ancient coins have been found to perform rather poorly. In their study, Arandjelovic and his colleague Jessica Cooper set out to develop a more effective approach, which can describe a coin like a human would to another human.
"The work with Jessica came from my realization that the field has been taking a very wrong angle of trying to determine whether two coins are the same," Arandjelovic explained. "The reason for this stems from the fact that few ancient coin types (relative to the tens of thousands minted during the five centuries of Roman Empire) which has been imaged is rather small, making the approach of little practical importance. Jessica and I thus thought that it would have been much better if the computer could describe the coin, much as a human would to another human."
"I'm broadly interested in algorithms that mimic the way humans approach tasks," Cooper told TechXplore" When an expert describes an ancient coin, she identifies artistically depicted concepts in the same way that our system does - by recognising shapes in the image. She is also capable of pointing to the elements she is describing: 'there is a cornucopia', 'there is a shield' etc. Our system also does this."
In-depth descriptions are a crucial part of numismatics literature, thus uncovering detailed information about coins using machine learning techniques could prove to be very useful. Coin descriptions are currently written by human experts, which can be quite time consuming. The new method developed by Arandjelovic and Cooper could help to speed up the analysis of ancient coins, automating a significant part of it.
"We use so-called Deep Learning, which uses a specific type of neural network (these are loosely - very loosely indeed - motivated by neural networks which comprise our brains) to learn from lots of examples of coins which do and which do not contain a specific visual element (e.g. shield, spear, etc.)," Arandjelovic said. "This is of course how humans learn in childhood: through repeated exposure and feedback from the supervisor (parent, teacher, etc.)."
The majority of existing approaches work by visually matching coins, using object recognition tools. However, the number of ancient coin types greatly exceeds the types of coins that have been recorded by experts digitally or on paper, which is the reason why these methods often perform poorly.
Unlike previous approaches, the method devised by Arandjelovic and Cooper analyzes the semantic content of coins. Firstly, the researchers used real-world multimodal input to extract and associate semantic concepts with the correct coin images. Subsequently, they trained a convolutional neural network (CNN) on the appearance of these concepts.
"Our most important result is the proof of concept, which we can reasonably expect to mark a turning point in the direction of the field," Arandjelovic said. "We already have a lot of new ideas as to how to improve on what we have done so far, and I trust that other researchers will be inspired to come up with different ideas that build on top of our contribution too."
The researchers evaluated their method on the largest existing dataset of ancient coins, which includes coin images extracted from 100,000 auction lots. Their tests yielded highly promising results, with their algorithm making correct associations and accurately identifying semantic patterns in ancient coins.
"I think our data is really interesting because it is challenging - there is a lot of class imbalance, a lot of noise, and the images are only labelled at the entire image level," Cooper said. "Therefore, during training, the model is told only if there exists a certain element on the coin, but not where it is - it must learn that for itself. Solving problems on difficult datasets like this is valuable not only for its own sake, but also because approaches developed for one use case can often be successfully applied across a variety of domains."
The CNN used by Arandjelovic and Cooper is loosely based on a renowned artificial neural network called AlexNet, which was originally used to classify photographs from the ImageNet dataset. According to Cooper, their study offers a clear example of how this kind of cross-pollination can prove very valuable. She is currently working on a project that applies similar computer vision techniques to cancer diagnosis from medical scans.
"We have several plans for future research," Arandjelovic said. "Firstly, we plan to directly continue this research, as we would like to have a system that quite literally describes an image of a coin, using full, proper sentences, just like those that you would see describing coins when they are sold at auctions. We would also like to develop methods that monitor online auction sites to detect stolen coins or fake coins."
Automatic attribution of ancient Roman imperial coins. DOI: 10.1109/CVPR.2010.5539841. https://ieeexplore.ieee.org/document/5539841
Reading ancient coins: automatically identifying denarii using obverse legend seeded retrieval. DOI: 10.1007/978-3-642-33765-9_23. https://link.springer.com/chapter/10.1007/978-3-642-33765-9_23
Ancient Roman coin retrieval: a systematic examination of the effects of coin grade. DOI: 10.1007/978-3-319-56608-5_32. https://link.springer.com/chapter/10.1007%2F978-3-319-56608-5_32
Towards computer vision based ancient coin recognition in the wild- automatic reliable image preprocessing and normalization. DOI: 10.1109/IJCNN.2017.7966024.
Ancient Roman coin recognition in the wild using deep learning based recognition of artistically depicted face profiles. DOI: 10.1109/ICCVW.2017.342. https://ieeexplore.ieee.org/document/8265553
© 2019 Science X Network