Intelligent camera app from Microsoft useful for people with visual impairment

(Tech Xplore)—For the blind and low vision population, technology can help them toward greater independence and quality of life. Microsoft researchers recognized the kinds of questions they might try to address with a supportive app.

How long do the cooking directions say I need to wait before this is ready? What denomination currency is being handed back? Where is my favorite brand soup on the shelf? Which door sign says Conference Room?

A Seeing AI app from Microsoft was unveiled on Wednesday and it goes in that direction.

The free app is for the blind and low vision community; it narrates the world around the person—those nearby, text, objects. It is designed to recognize faces of people you're with, as well as facial characteristics, approximate age, and emotion.

How it works—it uses the smartphone camera to tell visually impaired users what's in front of them. With this intelligent camera app, you just hold up your phone and hear information about the world around you.

If the person is shopping, for example, this is how it works, as explained in a video presentation. "Seeing AI plays audio cues to give you an indication of when it first recognizes the possible presence of a barcode. The faster the beeps, the closer you are. Then eventually you will hear a description of what the product is."

In sum, you can use barcodes to identify what the product is, just by holding the phone —rotating the object until you hear the beeps.

Seeing AI is an ongoing research project. It is available in the iOS App Store, but some may ask, well, this is Microsoft—why iOS?

Business Insider had this thought: "Embracing third-party platforms is a strategic play by Microsoft to stay relevant in the mobile ecosystem, which the company has struggled with."

Similarly, Jordan Novet on CNBC said "its arrival on iOS is the latest example of Microsoft, under CEO Satya Nadella, embracing third-party platforms.

Novet gave some more examples. "Microsoft brought its SQL Server database software to Linux, and it has ported Office apps to the iPad."

Mashable, meanwhile, actually tried out the app. Karissa Bell reported that analyzing text and barcodes is what the app did best.

"It was able to read just about every snippet of text we threw at it and identified most barcodes easily."(What happens is that it beeps when you get near to the barcode and will guide you t to line up the camera with the edges of the document. As for other functions, it was apparent to her in the tryout that Microsoft still has more work to do.

There were three customer reviews at the time of this writing on the app page. One of the reviewers praised how "quickly it computes and reads off information and words in real-time."

Another thanked Microsoft for an impressive application.

Another response was from a person who said the app was "amazing for blind and visually impaired users."

Business Insider commented on where this places Microsoft in visibility: "The technology—which uses computer vision to identify the objects in a user's environment to provide context on their surroundings—points to Microsoft's AI clout. It also helps to position the firm as a front-runner in the rapidly growing mobile health ecosystem."