This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:


trusted source


Sketch-based object detection tool using machine learning could boost tumor detection

Sketch-based object detection tool using machine learning could boost tumor detection
Qualitative retrieval results for cross-category FG-SBIR. Credit:

Teaching machine learning tools to detect specific objects in a specific image and discount others is a "game-changer" that could lead to advancements in cancer detection, according to leading researchers from the University of Surrey.

Surrey is set to present its unique sketch-based object detection tool at this year's Computer Vision, Pattern, and Recognition Conference (CVPR). The tool allows the user to sketch an object, which the AI will use as a basis to search within an image to find something that matches the sketch—while discounting more general options.

Professor Yi-Zhe Song, leads this research at the University of Surrey's Institute for People-Centered AI. He stated, "An artist's sketch is full of individual cues that words cannot convey concisely, reiterating the phrase 'a picture paints a thousand words." For newer AI systems, simple descriptive words help to generate images, but none can express the individualism of the user or the exact match the user is looking for.

"This is where our sketch-based tool comes into play. AI is instructed by the artist via sketches to find an exact object and discount others. Which can be amazingly helpful in medicine, by finding more aggressive tumors, or helping to protect by detecting rare animals."

An example that researchers use in their paper to the conference is of the tool helping to search a picture full of zebras—with only a of a single zebra eating to direct its search. The AI tool takes into account, such as pose and structure, but bases the decisions off the exact requirements given by the amateur artist.

Professor Song said, "The ability for AI to detect objects based on individual amateur sketches introduces a significant leap in harnessing human creativity in Computer Vision. It allows humans to interact with AI from a whole different perspective, no longer letting AI dictate the decisions, but asking it to behave exactly as instructed, keeping necessary human intervention."

More information: Paper: What Can Human Sketches Do for Object Detection?

Citation: Sketch-based object detection tool using machine learning could boost tumor detection (2023, June 21) retrieved 5 December 2023 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Computer sketches set to make online shopping much easier


Feedback to editors