Music with the blink of an eye

Aliyah’s first choice was the dog bark. Next, it was a bird trill. Watching her move back and forth in her wheelchair, tilting her head in front of a computer screen to trigger the sounds was like seeing joy full-on. During the performance, some of the children from the Mackay Centre School played hand drums or clapped and danced to the beat. And Aliyah, along with the half-dozen other children in wheelchairs, were able to join in thanks to laptop computers that allowed them to trigger sounds through movements that were sometimes so limited as to be barely visible, but that were still captured by the computer cameras.
Students at the Mackay Centre School in Montreal (including Aliyah, on right) use software to make music and improvise together. Eric Lewis, a professor in the Philosophy Department, brought the software for the school to try and out. It was such a success that further projects are likely to come of it. Meanwhile students at CIRMMT continue to work to improve the program.

New software allows the differently able to get creative

By Katherine Gombay

Aliyah’s first choice was the dog bark. Next, it was a bird trill. Watching her move back and forth in her wheelchair, tilting her head in front of a computer screen to trigger the sounds was like seeing joy full-on. During the performance, some of the children from the Mackay Centre School played hand drums or clapped and danced to the beat. And Aliyah, along with the half-dozen other children in wheelchairs, were able to join in thanks to laptop computers that allowed them to trigger sounds through movements that were sometimes so limited as to be barely visible, but that were still captured by the computer cameras.

The Adaptive Use Musical Instruments (AUMI) software was developed by a group that is interested in improvisation and music. Which is how McGill philosophy professor Eric Lewis got involved. He is the McGill Site Coordinator of the Improvisation, Community and Social Practice (ICASP) research group and first saw AUMI software in operation at an ICASP conference about two years ago. He knew immediately that he wanted to work with it. ”It was one of those penny-drop moments,” says Lewis, “because it had the whole combination of being simple and cheap and effective and therapeutic and creative as well as conceptually interesting.”

Lewis, whose teaching and research interests range from presocratic philosophers to the philosophy of music and aesthetics, has become increasingly focused on both the practice and the theoretical implications of musical improvisation in recent years.

“Improvisation raises questions about the source of our capacities and abilities,“ he explains. “We still tend to operate with a certain Cartesian model where the brain acts as central control and it decides to do things and moves the body around. But working both with this software and with people with disabilities, you realize to what degree this is a false model. A lot of work in cognitive science, artificial intelligence, philosophy and disability studies is making us realize that the divisions between mind, body and machine can be pretty porous and that’s philosophically interesting.”

And while it may be theoretically interesting, it is clear that the process of working with the AUMI software is as much fun for Lewis as it is for the children at the Mackay Centre School.

“I felt that I was at that stage in my career where I wanted some direct, obvious link between what I do and improving someone’s life in a very straightforward way,” Lewis acknowledges. “And so when I saw this project and the work they do with a kid who can barely move and interact with the world and suddenly they get this smile and start dancing in their wheelchair…. It’s a very direct link between what you’re doing and people’s lives.”

But while seeing the joy that disabled children get playing music for the first time is important to Lewis, what is just as important to him is witnessing how making music is allowing the children to participate actively in community making in a way that they haven’t been able to in the past.

“Community artistic practice is a way of codifying collective memories. But because most instruments are designed for a normalized body, like a violin, for example, which is built for people with two hands, people with non-normalized bodies have been left out of this crucial form of community building and community practice,” says Lewis. “The more limited movement you have, the more you are closed off from entering into these kinds of community activities. And since we know that to such a great degree our own sense of self is partially a product of how we relate to the community, this tool allows people with more limited movement to build a more enriched sense of self.”

It’s clear, from watching the children in wheelchairs at the Mackay Centre School using the AUMI system to make music with the other kids in the room, that the software is doing just this. At one point, one of the children dancing around and playing a drum went up to one of the kids in a wheelchair and gave him a high five – happy with the sounds they were producing together.

The software is available free, and has been developed to be run, with very little training, on the kind of aged computers that are likely to be found in schools and centres for the disabled. If a computer camera is sensitive enough, it can capture a movement as small as a human breath, or the blink of an eye, and use this to trigger sound. Depending on their taste, AUMI users can choose from a vast range of sounds, ranging from the clip clop of a horse to a blues riff on a piano, and can manipulate these sounds in different ways through their movements. The software is in constant evolution as users and therapists suggest changes to improve its accessibility, and Ian Hattwick, a PhD student at the Schulich School of Music, is currently updating the software to take in some of their suggestions.

Patrizia Cicarelli, the Principal of the Mackay Centre School watches the AUMI performance with excitement.

“The kids are really having fun… even some of shy kids I didn’t think would play were doing it. We’re definitely going to keep using it.” Cicarelli and Lewis have already started discussing how to move forward using the AUMI software. Lewis hopes that future research at the Mackay Centre School will go beyond the creative and community building aspects of the software to explore its therapeutic possibilities… to see how the software can be adjusted to encourage children to increase their neck rotation or fine motor skills, small movements that could make the difference between a feeding tube and a child’s ability to feed themselves. Music that makes a real difference to life in more ways than one.