2022-029 – Automatic Generation of Vocabulary from Photographs for Use in Augmentative and Alternative Communication and Language Therapy Systems

  • Generates vocabulary automatically, based on available and user-generated images
  • Automated: empowers users and provides new therapy paths to physicians and therapists
  • Mobile application: easy to deploy, no additional HW/SW needed

Advantages

This is a mobile application solution to be used as an Augmentative and Alternative Communication (AAC) device, or language therapy tool for non-verbal individuals, such as people with aphasia or in the autism spectrum disorder.

The system applies artificial intelligence on input photographs provided by the user to automatically generate a set of words and phrases related to the scene photographed. This vocabulary set is displayed as symbols, and reproduce computer-synthesized speech when selected, allowing people with language impairments to communicate about personally relevant activities or concepts depicted in the photographs.

To date, no other AAC system is able to generate vocabulary automatically, requiring language therapists or family members to manually pre-program the devices with relevant vocabulary, which consequently limits the support to basic generic expressions of wants and needs. Our system can also be used during language therapy sessions for teaching symbolic concepts and how to combine them to express ideas, aiming at facilitating the use of traditional symbol-based AAC devices.

Potential Applications

Assitive devices

Augmented Alternative Communication (AAC), therapy companion for non-verbal individuals (aphasia, authism spectrum disorder)

Contact Information

Name: Francesco Tordini

Email: francesco.tordini@mcgill.ca

Phone: 4388811973