Microsoft AI app helps visually impaired users to 'see'

Seeing AI app uses smartphone camera and AI analysis to provide audio description of people and objects

Tags: AccessibilityArtifical intelligenceMicrosoft Corporation
  • E-Mail
Microsoft AI app helps visually impaired users to 'see' The Microsoft Seeing AI app can recognise friends, and give an audio description of a person and their facial expression.
By  Mark Sutton Published  July 24, 2017

Microsoft has launched a new artificial intelligence-based mobile app to help visually impaired people to navigate the world.

The Seeing AI app, developed as a Microsoft research project, uses the camera on a smartphone with audio to help people to identify people and objects around them.

The app uses AI analysis of images and cloud connectivity to identify objects, documents, or people, and then instantly gives an audio description of the subject. Audio instructions help the user to align the camera with the object, while a host of features provide depth of descriptions.

The app is able to capture short text, and read it aloud to the user, while longer documents can also be scanned using character recognition. The app can also recognise currency notes of different denominations, including notes that are crumpled or partly obscured.

For people recognition, the app is able to recognise friends who have already been scanned, and also to scan and describe other people, including estimated age and their emotion or facial expression. The app has also been trained to recognise objects and activities to include those in the description of scenes.

The app can also provide product descriptions from barcodes, and includes audio cues to help a user to locate the barcode or image for the camera to capture the image.

The app is available for iOS at present.

Add a Comment

Your display name This field is mandatory

Your e-mail address This field is mandatory (Your e-mail address won't be published)

Security code