We recently completed a sequence of technical trials involving our OSDPI-implemented on communication board (CB) The goals of these trials were as follows:
- Work out video camera set up for assuring optimal viewing of both the subjects from waist up to enable us to see behaviors such as gestures, eye gaze, etc.
- Work out video camera set up for assuring optimal viewing of the CB display without being obtrusive.
- Assure our OSDPI-implemented CB is working optimally in actual conversational use.
- Try out eye-tracking glasses which we plan to use in future experiments. Assure the eye-tracking glasses are working and the output can be integrated into ELAN for analysis.
- Preliminary trials comparing use of the CB in Speech Generating Device (SGD) mode (with speech output and message window) and CB mode (analogous to using a non-electronic CB, but with the ability to log selections in OSDPI).
Camera Positioning:
We found that positioning the social camera as shown below provides the best view of both participants in two party conversations. This positioning also allows participants to view each other face-to-face while still allowing the typical speaking (TS) participant to view the message display and message window communication board in conditions where this variable is desired.
As shown above, the second camera was positioned with the tripod to the side of the CB user so as to be unobtrusive. We purchased a boom extension for the tripod to suspend the camera directly over the CB.
Robustness of OSDPI Communication Board in conversational use
During our preliminary conversational trials (described below), we had difficulties with one of the platforms we planned to use in upcoming experiments. We purchased a foldable Windows OS laptop [Dell Latitude 9520 – 2-in-1 Laptop] with a touch screen for this project. When it was used during our initial conversational trial, we encountered two problems. First, after a few minutes of use the touch screen became less responsive. Second, when the laptop was positioned at an angle so the TS partner could not see the screen, it was not stable. It moved around down when the user selected buttons vigorously and it moved around on the table which made it difficult to keep in view of the camera.
We had also purchased a Grid Pad 15 SGD for our lab. This device also uses Windows OS so it can run programs from OSDPI. This device worked successfully in the other preliminary trials. The Grid Pad 15 has a built-in stand for positioning at an angle, but can also be positioned flat. It did not bounce up and down when accessed or move around on the table. We plan to work to solve the issues with the laptop as this is a more cost effective device for implementing the CB in future studies, especially if we run multiple subject dyads simultaneously in different locations in the lab.
Eye-tracking glasses implementation
See see blog on OSDPI integration with 21st Century CB for details on integrating output with ELAN for details.
We learned that we need to screen the vision of our subjects to assure that the subjects in both the augmented communicator (AC) role and the TS role can see the CB without corrective lenses. Although our two pairs of eye-tracking glasses can be fitted with corrective lenses, this would not be cost effective or practical for future research trials. We also need to determine if our eye-tracking glasses will work with subjects who wear contact lenses.
Preliminary Conversational Trials–Three Conditions:
To begin to examine the effect of varying options on our CB on engagement, we ran three very preliminary trials.
Subjects:
We used two students in the lab for these trails, one student was assigned to be the AC and the other the TS. The student who played the AC received a 90 minute orientation session. During this time, she received instructions on how the CB worked and the rationale for the organization of the words, letters and other elements on the message display. She also practiced inputting 50 sentences constructed to develop familiarity with finding the pre-programmed words. The target words were highlighted in red to alert her to search for them rather than resorting to spelling. After the orientation session, the student was provided with a paper copy of the CB display and asked to practice inputting sentences for 15 minutes a day.
Instructions to subjects:
The subjects were instructed to have a conversation based on the following three scenarios, one for each trail. The AC was told to use the CB, but she could also use gestures with her hands or head (nods and shakes) as desired. The TS was instructed to engage naturally and do what she felt would be helpful during the conversation.
- The AC borrowed their friend’s car (friend played by the TS) to go grocery shopping and while they were in the store a shopping cart smashed into the driver’s side door causing extensive damage. The car doesn’t have collision insurance and the AC is aware of this.
- The AC just got engaged and drops by their friend’s house (friend played by the TS) to tell them the exciting news. The AC also wants to find out what is new and exciting in their friend’s life.
- The AC does not want to go to a party/out to eat/to a movie/to the library to study, but their friend (the TS) wants them to go. The TS is trying to convince the AC to go to the party, and the AC is trying to explain why they don’t want to go.
Trial 1: “SGD mode–No see”–In this trial, the participants were positioned as described above. We used Option/Version 2 of the CB set up in SGD mode which included the “speak” and “clear display” buttons. Speech output was set to speak the preceding message when the speak button was selected or when the user selected terminal punctuation (“.”, “?”, “!”). The CB was positioned at an angle so the TS could not see the message display or the message window.
The TS wore eye-tracking glasses in this trail. The AC did not because she could not see the display without her corrective lenses.
[insert video of Trial 1]
Trail 2: “SGD mode–See” In this trial the conditions were the same as for trial one except the CB was positioned so the TS could clearly view the message display and the message window. Neither subject wore eye-tracking lenses in this trial.
[insert video of Trial 1]
Communication Board used in Trials 1 and 2
Trial 3: “Communication Board mode” In this trial the CB was set up to emulate a non-electronic communication board. The message window and speech output were turned off. Logging was on, but this is transparent to the subjects. Neither subject wore eye-tracking lenses in this trial.
[Insert video of Trail 3]
Communication Board used in Trial 3
Next steps:
We are currently in the process of inputting the logging data from OSDPI and the eye-tracking glasses (when used) into ELAN as well as transcriptions of portions of the videos from the trials. This information as well as the other things we learned from our preliminary trials will be used to plan future experiments.
A preliminary viewing of the trial videos supports our working hypothesis that conditions which engender joint attention and responsiveness and adhere to the shorter turn taking delay–closer to the expected delay seen in typical conversation, show more engagement. The continuum of most to least engagement appears to be CB>SGD-See>SGD–No see.