21st Century Communication Board

Project Description

Co-Directors: Higginbotham, Mathy
Supervising Researchers: Higginbotham, Mathy, Bizovi
Associate Researchers: Satchidanand, Golleru, Buckley, Hutchinson
Start Date: September, 2022

In this project we are re-imagining the traditional “low-tech” communication board (i.e., word and alphabet board) as an interface on an e-paper hardware platform. Without a doubt developments in assistive technology over the past 50 years including the alternative access techniques, speech generating devices (SGDs) and alternative computer access has greatly improved the lives of people with complex communication needs who rely on augmentative communication. However, research examining the use of SGDs during face-to-face interaction has identified issues related to engagement and staying in-time with the conversational flow (Higginbotham and Wilkins, 1999). On-the-other hand, interactions where the augmentative communicator (AC) uses a communication board show greater engagement and in-time synchrony. The development of an e-platform allows us to study the details of the co-constructive manner in which communication board conversations are transacted, and will allow us to explore ways of optimizing this technology to maintain engaged social interactions while benefiting from advanced interface features such as NLP, dynamic displays, and conversational speech synthesis.

Background

The earliest known communication board that was generally available was the F. Hall Roe Communication Board, developed for and with F. Hall Roe, who had cerebral palsy. 

Through the generosity of a local men’s group in Minneapolis, these boards were printed on Masonite and had notches that allowed them to be mounted between the arms of a wheelchair. They included both the alphabet and commonly used words and were used both by individuals with cerebral palsy and others in hospitals who were temporarily unable to speak. (Vanderheiden, 2002).  

With advances in “high-tech” augmentative communication devices that provide text-to-speech synthesis, currently referred to as speech-generating devices, low-tech communication boards are often considered as “back-up” communication tools to use in case the SGD is not available. However, a closer look at the role of communication boards reveals that some augmented communicators (ACs) use them strategically and effectively based on the communication context. For example, the communication board pictured below was designed by Jane D, an AC who also uses a speech generating device. With her communication board (see below), Jane was able to communicate at an average rate approaching 20 words per minute (wpm) when interacting with communication partners who could read and follow her rapid pointing (Higginbotham and Wilkins, 1999). With partners who could not do so, she used her speech generating device, however, with a considerably reduced rate–averaging less than 7 wpm which is the typical rate reported for ACs using speech generating devices. Due to her medical needs, Jane lived in a skilled nursing facility where she encountered communication partners with a range of abilities and constraints. She selected between her communication board and her SGD based capabilities of her communication partners as well as the requirements of the communicative activity. She preferred using her communication board for face-to-face conversation when possible because of the engagement and the ability to communicate more efficiently. 

Dr. Richard K. Olney, founding director of the ALS Treatment and Research Center of University of California San Francisco, and a person with ALS uses a low-tech communication board similar to the one pictured below offered by LowTech Solutions, accessed with an adapted laser pointer for most face-to-face communication because he finds it faster and simpler to use than his SGD. He uses an adapted computer for emails and writing medical articles (https://store.lowtechsolutions.org/testimonials/). 

How do communication boards enable ACs to communicate in what both users and communication partners perceive as a faster, simpler manner? Conversation analytic research with ACs and their mouth speaking partners has found that communication boards facilitate engagement by providing a platform where partners continuously coordinate in co-construction of the AC’s conversational contributions (Higginbotham, Mathy & Yoder, 1988; Higginbotham and Wilkins, 1999). For example, in this short video of Dr. Olney using his communication board, accessed with a laser pointer to talk to his wife https://www.youtube.com/watch?v=hEtsslVILGE&t=1s, notice how both partners are highly engaged throughout the message construction process. Also notice that because the partner must continually attend as the message is being constructed, she is often able to anticipate and offer candidate completions of message parts (i.e., words, phrases) for confirmation. Utterance length tends to be shorter but communication rate is faster likely due to co-construction. 

In contrast, SGD’s do not require the partner to attend to the message composition activities of the AC until the utterance is spoken via synthesized speech. As seen in the following exchange between S who uses a scanning SGD and her sister, S spends over three minutes composing a sentence in response to their earlier conversation. During this time R discusses two additional topics while R is vocalizing (not shown), nodding and simultaneously composing her utterance. When S finally speaks via synthesized speech, R doesn’t understand it, requiring S to make at least two  attempts to repair her misunderstood utterance:

S (device user w/ALS) & R(sister, conversation partner)
(From Koroschetz, J & Higginbotham, J. (in progress). Subject 05.)

R: You know, can you when you write something there…  Can you print out the text of what you write?

S: ((shakes head/starts scanning to compose))

((3 minutes of R’s conversation omitted. S continues to compose her utterance via scanning during this time))

R: … We’ve been reading the acts of the apostles and there’s a chapter in there on anointing not only used in the church for healing but through the Apostolic succession, and the ordination of the priest that is needed it, in that it is Christ who anoints.

(9sec)

R:  Are…

S: ((Looks toward R, nods)

R: Yea, we- talked a lot about that last week,

S: ((Nods head))

R: Of course that’s true of all the sacraments. [and uh

S: [I wonder how Rita and her family are. (1 sec)

R: Who? ((leans forward)) who in the family?

S: vhriahh ((attempts to say ‘Rita’)  ((S looks at R then to the computer display))

R: I didn’t get the computer lady’s pronunciation of the name

S:  ((restarts scan))  (2.4sec).   I wonder how Rita and her family are. 

R: How Rita. I know! I thought of them in this story.

Work So Far

Efforts this year are directed at developing a prototype communication board and implementing it using Project Open’s Open-Source Design and Programmer Interface (OSDPI) as an initial research platform.  We are planning to start running research trials examining the value of engaged co-construction during AAC-mediated conversation starting in spring 2023. 

From fall 2022 to winter 2023 we worked on our initial prototype. Primary considerations for developing the communication board (pictured below) included determining the dimensions of the display and the number of cells as well as the specific words and phrases to include. Our initial prototype was designed to be implemented on a 13” to 15” touch screen on a tablet computer. To allow sufficient space for the AC to make selections and sufficient visibility for the communication partner, we settled on a 9 by 12 layout for a total of 108 cells. 

In the prototype we removed the visible grid (cell borders) to resemble the original Roe Hall communication board, however, the grid can be easily replaced with OSDPI. Also, similar to the Roe Hall board and to facilitate easily viewable letter and word selection, we shifted the letters to the left or right in the rows and we placed the words in the upper section of the cells. 

Vocabulary selection for our prototype was based on an analysis of the Roe Hall communication board as well as currently available low tech boards from LowTech solutions (https://store.lowtechsolutions.org/). Based on their size allowances, these boards include various numbers of frequently used words (i,e., pronouns, auxiliary verbs, verbs, prepositions, conjunctions, determiners, etc.) that make up the highest proportion (60% – 80%) of spoken English vocabulary, referred to in the AAC field as “core vocabulary”. On our prototype, with some exceptions, the vocabulary was arranged grammatically from left to right with items within grammatical categories alphabetized. Color coding was used to distinguish the various word groups. 

Similar to traditional low-tech communication boards, we included only terminal punctuation. A “space” cell was included as an option for the AC to indicate the end of words. Our initial prototype includes only two conversation management messages, “Wait” and “Start Over”, however, we are developing a bank of additional conversational management messages that will be included on communication boards in future research trials.

The communication board was designed with the option of toggling on and off a message window, which is off in the picture below. There are also empty buttons in the top row that we are considering using for topic relevant vocabulary as we run research trials. 

The OSDPI programming environment (shown below) not only allows for highly flexible programming of augmentative communication applications, but it is designed to enable data logging, by tracking the time and the cell selection across time. In our research trials we will use both data logging and analysis of videos to examine interactions using our initial prototype and future versions of our 21st Century Communication Board. 

Higginbotham, D. J., Mathy-Laikko, P., & Yoder, D. (1988). Studying conversations of augmentative communication system users. The vocally impaired: Clinical practice and research, 265-294.

Bibliography

Higginbotham, D. J., & Wilkins, D. P. (1999). Slipping Through the Timestream: Social Issues of Time and Timing in Augmented Interactions. In D. Kovarsky, M. Maxwell, & J. F. Duchan (Eds.), Constructing (in) competence: Disabling evaluations in clinical and social interaction (pp. 49–82). Psychology Press.

Vanderheiden, G. (2002). A journey through early augmentative communication and computer access. The journal of rehabilitation research and development. 39 (Supplement) pp. 39-53.

Blog Posts