The touch of technology

Vincent Levesque's STReSS2 tactile display renders graphics "seen" by the user's fingertips. Photo: Owen Egan.
Vincent Levesque's STReSS2 tactile display renders graphics "seen" by the user's fingertips. Photo: Owen Egan

By Mark Shainblum

We talk about having five (or six) senses, but there’s no escaping the fact that we interact with the world primarily through sight and sound. The very words for their loss, “blind” and “deaf,” are buried deep in the language. Can you imagine anyone ever saying “I swear, Nancy is so ansomic! Can’t she smell that Brad’s madly in love with her?”

The dominance of seeing and hearing over the other senses affects everything humans create, from art to architecture to technology. A super-intelligent dog – like the title character in Olaf Stapledon’s 1944 novel Sirius – might invent an art form based entirely on smell, but the idea would never occur to a human being. Similarly, we interact with most of our technology through sight and sound. By definition, a computer monitor must be a screen that transmits information to your brain through your eyes, right?

No, not really, say the diverse team of researchers at McGill’s Haptics Laboratory. The lab, a division of the Centre for Intelligent Machines at the Faculty of Engineering, is based on the fourth floor of the Wong Building.

Haptics, or “touch-based” technologies transmit information to the user by applying forces, vibrations or motions. The first true haptics were developed in the 1940s in the form of aircraft “stick shakers,” which transmitted vibration to the pilot’s hand to warn of an impending stall. By the 1950s, remote-controlled robotic arms designed to handle radioactive substances used force feedback to provide a rough simulation of the sense of touch. Today, digital force-feedback joysticks and controllers – immeasurably more sophisticated than those early mechanical dinosaurs – are standard video game accessories. Many games are so dependent on the sense of touch that they can’t be played without these haptic add-ons.

Akihiro Sato’s “morpheotron” might sound like it belongs in a video game or a manga published in his native Japan, but please get the image of 100 foot tall warrior robots out of your head. The sophisticated mechatronic device actually “displays” shapes through the dual senses of touch and body position.

“You can program it to display any shape,” explained Sato, who just earned his Master’s degree in Engineering. “In the morpheotron, your finger rests on an incline plate that rocks back and forth as the platform it’s on moves up and down. As your finger is pushed in these multiple planes, you feel the programmed shape.”

“The morpheotron can theoretically be used for artificial perception in a virtual environment,” Sato said. “But here we’re doing more basic research. We want to know how the brain integrates the information we give it through the finger, including height and inclination, and we don’t know which of these is more important.”

The small, cube-shaped device hooked up to Vincent Levesque’s computer might be mistaken for an old first-generation mouse, but it’s not. It’s the STReSS2, a tactile graphics computer monitor that you read with your fingertips, not with your eyes. A so-called “lateroteactile” display, the Stress2 stimulates the fingertips with a matrix of 64 metal pins that move in multiple directions. These pins – or actuators – are made of a piezoelectric material that bends with the application of electric current, so each one can be controlled and moved individually. It produces four different tactile sensations which can be combined to generate icons, charts, maps and other refreshable graphics the user “sees” through her fingers.

“By compressing and stretching your skin, it creates the illusion of a 3D surface under your finger,” explained Levesque, who recently completed his PhD studies. “This was designed for visually impaired children in school, who need the same information as the other students to learn. They also, very simply, want to be able to communicate with their sighted friends using the same information.”

Hsin-Yun Yao waves a long black tube that looks like nothing so much as a magician’s wand. And so it is, though not in the popping-a-rabbit-out-of-a-hat way. The “virtual rolling stone” is an experimental device which electromechanically simulates a free-rolling ball inside a hollow tube. As you rock the tube back and forth, you are absolutely convinced that there is a heavy metal ball-bearing rolling around inside it, but it’s all an illusion. There’s no ball-bearing, and the tube isn’t hollow.

“The main purpose of this experiment is to study psychophysics,” said Yao, another recent McGill PhD. “How do we perceive gravity? Is it something we have modelled inside, or is everything we do based on timing? That’s what we are trying to determine.