TAI-CHI "Tangible acoustic interfaces for computer human interaction"
Research Area:
Research Lines:
Research Lines:
Responsible:
Research
-
Start date: 2004-01-01
Length: 36 months
Project abstract
It is an everyday experience to use tangible input interfaces (keyboard, mouse, touch pad, touch screen, etc.) to operate with any kind of electronic device, from interactive kiosks to digital personal assistants or bank machines, but it is unusual to control common objects (doors, windows, tables, chairs, etc.) with a tactile interaction (for example, with a finger touch).
This inability is mainly due to the fact that the available computer-human interfaces are based on active methods to acquire the information induced by the human touch: a field (electric, magnetic, infrared, etc.) is induced and the variation of the field properties due to the tactile interaction is read like a source of command information.
The active methods require the use of electronic devices (switches, potentiometers, sensitive layers, force resistive sensors, etc.) that limit the interaction to predetermined points or surfaces, that increase the manufacturing costs and that make the solutions not feasible to the majority of the applicative contexts. We here propose a different technology based on a passive method (the source of the information is the tactile interaction) and able to transform the object to control into a tangible interface. The project is ended and it received the IST Best Exhibition Award for the 2006 at Helsinki.
This inability is mainly due to the fact that the available computer-human interfaces are based on active methods to acquire the information induced by the human touch: a field (electric, magnetic, infrared, etc.) is induced and the variation of the field properties due to the tactile interaction is read like a source of command information.
The active methods require the use of electronic devices (switches, potentiometers, sensitive layers, force resistive sensors, etc.) that limit the interaction to predetermined points or surfaces, that increase the manufacturing costs and that make the solutions not feasible to the majority of the applicative contexts. We here propose a different technology based on a passive method (the source of the information is the tactile interaction) and able to transform the object to control into a tangible interface. The project is ended and it received the IST Best Exhibition Award for the 2006 at Helsinki.
Project results
ISPG prototype
A prototype system that transforms common thin panels into tangible interfaces was developed by the ISPG (Image and Sound Processing Group) research group at the Politecnico di Milano. The user can interact with his fingers or with other tools (wooden stick, pen, screwdriver, chalk stick, etc.) on these plates and the system localize and track the position of the interaction. Some “virtual boards” are therefore developed with thin panels of common materials (MDF – Medium Density Fiberboard, Plexiglass, glass, etc.). The user can write or draw on the panel and the result is displayed in real time on a screen or saved in the memory of a PC. The algorithm of localization and tracking of the finger touch is going to be protected by a patent copyright. The developed technology, or its extension to any shape objects, is really interesting and there are many possible applications not only in the fields strictly connected with the informatics or the electronics sciences, but also in all those whose aim is to make automatic the interaction between a man and a common object at low manufacturing costs.
Public events
Summary of major technical achievements
Publications
A prototype system that transforms common thin panels into tangible interfaces was developed by the ISPG (Image and Sound Processing Group) research group at the Politecnico di Milano. The user can interact with his fingers or with other tools (wooden stick, pen, screwdriver, chalk stick, etc.) on these plates and the system localize and track the position of the interaction. Some “virtual boards” are therefore developed with thin panels of common materials (MDF – Medium Density Fiberboard, Plexiglass, glass, etc.). The user can write or draw on the panel and the result is displayed in real time on a screen or saved in the memory of a PC. The algorithm of localization and tracking of the finger touch is going to be protected by a patent copyright. The developed technology, or its extension to any shape objects, is really interesting and there are many possible applications not only in the fields strictly connected with the informatics or the electronics sciences, but also in all those whose aim is to make automatic the interaction between a man and a common object at low manufacturing costs.
Public events
- Development of a user interface for the museum installation in Como (flash application embedded in a website, which incorporates games, quizzes, navigation tools, etc.).
- Development of a user interface for a live performance at the Opera Theater in Como (November 3, 2005). The Concert was organized for the 2005 Conference of the Audio Engineering Society, Italian Section. The interface allows a drummer to design his/her own percussion set in a graphical fashion and to specify physical parameters such as stiffness, density, losses, etc. When in “performance mode” the drummer plays the percussion set using a TAI specifically developed for the event.
- Development of the HW structure of MUST-RD (MUlti-Sensor Tracking through the Reversal of Dispersion): a real-time system for touch localization and tracking in thin plates.
Summary of major technical achievements
- In-depth study of elastic wave propagation in thin plates and of dispersion phenomena.
- Development of estimation techniques for active (based on signal injection) and passive (based on finger touch) estimation of the elastic properties of the board.
- Development of a technique to estimate the finger touch signature.
- Modeling of elastic wave propagation in thin plates, taking into account border reflections through real-time acoustic beam tracing.
- Finger touch localization and tracking in thin plates by using the inverse problem theory.
- Multi-sensor signal management based on inverse theory, aimed at improving the robustness of touch localization in the presence of undesired scatters.
Publications
- A. Crevoisier, P. Polotti, "Tangible Acoustic Interfaces and their Applications for the Design of New Musical Instruments", 2005 International Conference on New Interfaces for Musical Expression (NIME 2005), Dublin.
- D. Rovetta, A. Sarti, S. Tubaro, G. Colombo, “Modelling Elastic Wave Propagation in Thin Plates“, 2006 International Conference on Intelligent Productions Machines and Systems (IPROMS 2006), Virtual Conference.
- G. Ribay, D. Rovetta, S. Catheline, A. Sarti, G. De Sanctis, “Modelling Wave Propagation in Thin Plates for the Localization of Tactile Interactions“, 2006 International Congress on Sound and Vibration (ICSV13), Vienna.
- G. De Sanctis, D. Rovetta, A. Sarti, G. Scarparo, S. Tubaro, “Localization of Tactile Interactions through TDOA Analysis: Geometric Vs. Inversion-Based Method”, 2006 European Signal Processing Conference (EUSIPCO 2006), Florence.
- D. Rovetta, A. Sarti, G. De Sanctis, M. Fabiani, “Modelling Elastic Wave Propagation in Thin Plates”, 2006 European Signal Processing Conference (EUSIPCO 2006), Florence.
