Home / / Microsoft AI app helps visually impaired users to 'see'

Microsoft AI app helps visually impaired users to 'see'

Seeing AI app uses smartphone camera and AI analysis to provide audio description of people and objects

The Microsoft Seeing AI app can recognise friends, and give an audio description of a person and their facial expression.
The Microsoft Seeing AI app can recognise friends, and give an audio description of a person and their facial expression.

Microsoft has launched a new artificial intelligence-based mobile app to help visually impaired people to navigate the world.

The Seeing AI app, developed as a Microsoft research project, uses the camera on a smartphone with audio to help people to identify people and objects around them.

The app uses AI analysis of images and cloud connectivity to identify objects, documents, or people, and then instantly gives an audio description of the subject. Audio instructions help the user to align the camera with the object, while a host of features provide depth of descriptions.

The app is able to capture short text, and read it aloud to the user, while longer documents can also be scanned using character recognition. The app can also recognise currency notes of different denominations, including notes that are crumpled or partly obscured.

For people recognition, the app is able to recognise friends who have already been scanned, and also to scan and describe other people, including estimated age and their emotion or facial expression. The app has also been trained to recognise objects and activities to include those in the description of scenes.

The app can also provide product descriptions from barcodes, and includes audio cues to help a user to locate the barcode or image for the camera to capture the image.

The app is available for iOS at present.