I am currently working on an assistive technology project that provides sound-based encouragement for expression/intentional communication through touch for non-verbal individuals. Kate Hartman, Fiona Carswell, Alice Tsang-Planas, Megan McMurray and Nanna Haalinen and I have been observing a young woman with Rhett’s Syndrome.

• Keys Design Elements as Described by Therapists: Repetition/Reinforcement, Structure/Pattern, Observation/Documentation, Audio feedback
• Working Project Title: Audio Quilt
• Population/Audience: Girl’s with Rhett’s Syndrome or anyone who functions with Pre/Non-Verbal Communication
• Need Addressed: Sound-Based Encouragement for Expression/Intentional Communication through Touch
• Description: A hanging large-scale quilted textile, with interchangeable (content) quilted sections that respond to touch with audio feedback. This can be used to develop interest and intentional response to narratives and other lessons which are already being used by therapists. This can be attached to a wall or draped over any surface, so that it can be used standing, seated or laying down. Unique audio feedback will be location specific to the individual sections that are triggered by the users touch.
• Technologies Used: Simple switches will be quilted into large areas of fabric that will trigger sound feedback from an isd chip. We are choosing to use sound feedback as our target user seems especially responsive to this. Also, the device is to be used from a standing or seated position depending upon the constant fluctuations of her physical needs.
• Materials Used: Fabric, conductive fabric, copper mesh switches (homemade), thread, conductive thread, snaps, 2500 or 4000 series isd chip(s?), velcro for interchangeable content.

Many thanks to Michael Schneider for his patient assistance getting the ISD chip working on our circuit board. We were able to successfully record and playback several voice recordings, using a very cheap ($2) microphone and in-chip pre-amp. This week we are reading through data sheets for this chip, the 2500 series and the 4000 series for their suitability for this project. Currently, we are interested in the user’s ability to record and save several (10) separate ‘lessons’ or narratives that can then be toggled through or selected and used for reinforcement.

This chip seems ideal in the following ways:

• cost
• multiple address pins
• push button recording and playback–simplicity of use
• size/weight
• ease of use, lack of need for training or updating

current documentation on isd chip 2590

Next Steps

We made a mock-up of version of the quilt and brought it to our target user to gauge her responsiveness to haptic and aural feedback. The quilt consisted of 6 separate switch sections, each made of a very different textural and formal configurations:

1. pillow switch embedded into foam
2. two pieces of acetate with thin foam tape separating them
3. a large ‘Enabling Devices’ switch inside of a denim patch
4. a silver dollar-sized switch inside of foam
5. two more patches that were meant to gauge the variations in touch, one overstuffed with styrofoam beads and one of very thin batting.

Materials Used:

1. denim
2. polar fleece
3. sueded polyester from baby’s pillow stuffed with styrofoam
   *make sure to check for washability and allergies

We also experimented with types of audio feedback. We set up two scenarios and also tailored our test to the user’s current physical state, that day she was tired from working with her physical therapist and from seizures earlier in the day. We re-grouped and tested the quilt on her lap and on the floor (skipping the standing version).

For the first scenario audio from a recorded session with the users’s speech therapist was triggered by the selection of velcro cards (the visual accompaniment to that session) which were attached to sections of the quilt. When the user selected a card by touching one of the sections we would fake the response by playing a corresponding audio track from an ipod. This was specifically intended to gauge the user’s response to recorded audio of someone that she knows and consistently ‘learns’ from. We were thrilled to see the user light up with recognition when she heard the voice. Although at first the user’s physical therapist guided her hand to the quilt sections, eventually, using her hand, she was able to consciously select two ‘correct’ responses independently, identifying a taxi and a subway as she was prompted by her ST’s voice recording.

The second scenario we placed velcro images of the users favorite movie, The Little Mermaid, on separate sections of the quilt as choices that would play certain songs and would continue the song beyond a 2-3 second play time. The user was very responsive to the images, beginning to clap at first sight. On her own, she made immediate choices and began dancing when the music played. After the first song stopped she immediately reached out and selected another section to trigger the music to begin again. There was a clear understanding between touching the quilt as a means to create a desired response.

First User-Testing Conclusions:

• Audio recordings by known individuals are recognizable and stimulate responses, reinforces lessons and stimulates conscious decision making, while being fun!!!
• The user did not like the recognizable ‘clicking’ switch feedback, preferring soft switches and immediate audio response.
• More specificially, the user appreciated the variation in the density and texture of the switches (or patches of the quilt). Eventually, she chose to use her nose to touch the switches, something that the user’s mother says she prefers to use sometimes–preferring the more elucidated feedback.
• When the user became physically tired and the quilt was placed on the floor, without instruction she intuitively used her feet to select sections of the quilt and continued to dance in response to the music.
• The user prefers immediate audio feedback to selections, otherwise she was likely to try the switch again.
• The user favors the left side of her body and has a tendency to want to bring her right hand to hold her left hand. Often times her therapists will hold onto her left hand, forcing her to direct her choices through her right hand independently. For this reason switches on the right side of the quilt should be more sensitive to touch as it takes a great deal more effort to even make it 6 more inches to the right of the center of her body.
• We need to be sure not to over-stimulate the user as this can bring on seizures which are currently frequent. This seems not to be such a problem with recorded voices from therapy sessions and more relevant to playing music (not the project’s intention).

We concurrently experimented with several switch types:

• 2, 10in square plexiglas plates with copper leafing and foam tape in between
• result: too rigid and heavy, labor intensive
• 2, 10in square acetate sheets with copper leafing and foam tape in between
• result: too flimsy, labor intensive
• 2, copper mesh pieces with foam tape in between
• result: lightweight and highly responsive, cheap, easy!

We plan to try a finer copper mesh, similar to a fabric. Actual conductive fabric doesn’t conduct enough.

next up:

1. Decide on chip, order/wire/test
2. Create final versions of switches with appropriate copper wire mesh, conduction thread and snap mechanism.
3. Decide if quilt patches are removable

» Projects