“Seeing AI”: A New App from Microsoft

While the number of apps for folks with visual impairments that use Artificial Intelligence (AI) and the ubiquitous “cloud” continues to grow, Microsoft’s latest entry, Seeing AI, may be a game changer.

According to Business Insider magazine, Microsoft first showed off the prototype over a year ago, but released the official version this week.

According to Microsoft, the new app “[…]narrates the world around you. Designed for the low vision community, this research project harnesses the power of AI to describe people, text and objects.” The software currently has five functions, called “channels.” Two channels deal exclusively with the recognition of text (one for signs and short phrases, and one that reads whole pages of text). One of the channels can detect and decipher bar codes. Another channel will soon be added that will be a currency reader.

The last two “channels” are perhaps the most exciting.

One channel, “Person” is able to detect the presence of a group of people, or individual persons by gender and also states the expression on their face. With some training and input from the user, the app has the capacity to facially recognize people and name them specifically, even if they are in a group.

The other exciting channel, “Scene” is still in beta but can identify other scenes and give its best guess as to what is happening. A test made by the author while sitting in the living room resulted in the app stating “man sitting on living room sofa.”

Designed for the iPhone, the app can be used with iPads or iPods but will need an active WiFi signal. The app is currently only available for iOS devices (no word on an Android version). Ironically, the announcement of the new app comes soon after Microsoft announced the end of the Windows Phone OS.

Look for significant improvements in the future. Microsoft is eager for feedback from users who can help refine the AI and accuracy of the application.

Microsoft’s Seeing AI website Download link in iTunes Watch Microsoft’s Seeing AI video

Thanks to John Brandt at MaineCITE for this post on Microsoft’s Seeing AI app for persons with visual impairments. The app is free and part of an ongoing research effort.

This post was first published as, “Amazing new app for folks with low vision” at MaineCITE on July 14th, 2017 and is reposted here with permission.