(Tech Xplore)—A trio of researchers in China has developed a system for measuring the intelligence of AI systems. They have also created a classification scheme meant to rank different systems. Feng Liu, Yong Shi and Ying Liu report in their paper uploaded to the arXiv preprint server that Google's AlphaGo system currently rated the highest of all those tested.
Scientists and average citizens alike have expressed fear regarding AI systems, some going so far as to suggest they may lead to the elimination of humans as seen in sci-fi movies. No one can prove one way or another whether we are headed toward an untimely demise at the hands of our own creations, but the researchers with this new effort contend that at least we can monitor the progress being made in machine intelligence. Doing so could alert us, they point out, to potential trouble if the machines start to score higher than humans on intelligence tests. To that end, they have come up with a seven-category classification system, ranging from first grade to seventh grade. Each category defines characteristics that a machine must have to be so classified. Using their system, the trio has classified AlphaGo as third grade, which, they note, falls below the intelligence of the average six-year-old human.
The system also produces a score which the trio claims, can be compared to scoring used for human IQ tests. They report that AlphaGo received a score of 47.28, which suggests an IQ of the same number. The average six-year-old has a score of 55.5. Meanwhile, Siri got a score of 23.9, Bing got 31.98 and Baidu got 32.92. It is no surprise to the scientific community that AlphaGo scored highly, as it was the system that beat the human champion Go player earlier this year.
The researchers also report that all of the systems they tested are getting smarter—each scored higher than they did on similar tests conducted two years ago. They also point out that their aim in creating a system for rating artificial intelligence is not to downplay their smarts, but instead to highlight achievements to date—rating the best as being on a par with the average six year old, for example, puts them ahead of most, if not every other creature on the planet, after all.