YMYI (You Move You Interact) is an interactive installation, where one is supposed to build up a body language dialogue with an artificial system so as to effectively achieve a synchronized performance between the real user´s body and the virtual object itself. The project aims at exploring a spatial sphere,where the user/performer is invited to develop his own creative inspiration based on his own body gestures and movements.
During the whole time assigned to research related to both the development and the scientific foundation stone of the prototype itself, we realised, in so far as the expectations brought on the user of the YMYI platform were concerned, that we were dealing with two key conceptions - narrative and image. Underlying these concepts, in-between the dual dimension of the human body and its perception of itself and the surrounding environment,we truly believe that the definitions given on this subject by the scientist Antonio Damasio constituted a resourceful enlightement to the scope of our investigation.
According to Damasio, on the one hand "the images (mental patterns) may be conscious or unconscious (...) The unconscious images are never directly accessible. The images access is to be provided in a single first person perspective ( my images, everyone´s images). On the other hand, the neural patterns are to be provided in a third person perspective. If I considered the possibility of observing my own neural patterns resorting to advanced technology, I would be always doing it in the third person perspective." ( DAMASIO, 2000:362).
We have the pleasure of leaving you an excerpt of his book "The Feeling of What Happens", which sums up the idealised proposal to the users of the YMYI artefact:
"It narrates a story, the story of a living organism unexpectedly astonished by performing its own change status while representing an object. However, what strikes the most is the fact that the knowing entity responsile for the act of astonishing is only created throughout the astonishment narration process."
(Damasio: 2000: 202.)
(1) DAMASIO, ANTONIO R. (2000). The Feeling of What Happens. Body and Emotion in the Making of Consciousness.
A Harvest Book
Experiencing the YMYI artefact
The YMYI artefact proposes a body dialogue between a user and a digital system, based on a live performance with gestures and movements within an interactive space. Thus, YMYI aims at exploring a space where a user develops an ongoing creative process duly synchronised with his own motion actions.
By entering a so-called “stage”, where the YMYI interactive installation takes place, the user meets, at a first glance, a whole regiment of particles flying in diverse directions and speed throughout the digital system. Within the experiencing sphere of the interaction itself, the user sees his own anatomic silhouette embodied in the canvas, in the shape of lines and curves. Meanwhile, the user realises that his gestures and movements, actually synchronised with his own physical displacement within the interactive “stage”, migrate in real time to the canvas. The performance acquires new shapes and contours. The space opens itself to tridimensionality. By moving leftwards or rightwards, upwards or downwards, the displayed interactive “me” goes hand in hand with the user´s own theatrical actions. The system becomes a stage which lets the emotion and the creativity come in. The living experience is unique and subjective.
The dramatic emphasis of the performance deepens: gestures embed themselves in sounds created by the body, like breathing, walking, standing up, hiding, as if we one was dealing with a contemporary dance performance.
The huge number of particles moving around the user fill in the scene background. They seem to have a life of their own, featuring living, dynamic and electrifying properties. Such particles tend to concentrate themselves in the edges of both the superior and inferior limbs. Feet and hands become a converging magnet able to attract and join them in a collective “bubble”. It is as if they behaved as a physical extension of the user´s body or embodiment. Thus, both particles and user form a single, united element, shining glamorously throughout the interface.
In this interactive scope, the user realises that light/soft movements bring on greater interactivity and emotion. Such an experience is “affordable” by the natural convergence of the whole particles throughout the given parts of the user body as a result from his gestures. If we try moving the left hand or the right one, or even the left foot or right one, the particles flow to each one of their edges accordingly. If we try to put our hands or arms in a downwards position, the particles flow upwards until they reach an upper position – the user´s own head. The interaction is experienced in a richer, more creative and more artistic way.
Another outstanding feature belonging to the interactive experience is actually the sound. In the YMYI artefact, we experience a synchronising sound reactive to the gestures dynamics of the user himself. The sound emits low in pitch or sharp frequencies, responding in real time to the user´s position, whether he is standing (upwards) or sitting/hiding ( downwards), respectively. Right on the top of the canvas, a sensitive sound to the touching facility of the user has the ability to project a sonorous lightning flash whose energy goes through the inner self.
In the artefact YMYI, the sound is a sole element which travels in harmony through the flowing gestures of the user within the interactive space so as to provide guidance and “affordance” to the user himself, in its artistic dimension. If the user stops moving, the sound concentrates in itself an ongoing powerful energy which provides a growing amplitude, whether in low pitch or sharp frequencies, which on their turn, become reactive to and at the same time dependent on the inferior or superior position of the user´s embodiment limbs, respectively.
The sound provides a multimodal enrichment to the whole interactive experience and creates by itself together with image and movement an interactive guide to the appropriation, exploration, knowledge and live experience of the artefact.
Human-computer interaction and enlightenment in the YMYI project
The Project You Move You Interact magnifies an essential keystone within the human-computer interaction science: the interactivity itself, which in this particular project achieves a high standard of success in performing an evolving artistic interface design generated from affordances, interconnected with the relational actions between a user and a digital system. Such affordances are neither explicit nor visible in the interface. They result, otherwise, of an input based on the exploration/discovery of an actor´s body movements in an interactive “stage” as well as of an output based on his embodiment display in an interface full of small particles, which energize a close communication link with the user himself, either by moving towards him, or by concentrating themselves on the edges of both his superior and inferior limbs.
As for the interface design in the context of usability is concerned, we can actually acknowledge that the project YMYI fulfills two main outstanding heuristic rules: the visibility and the consistence of the whole system. The visibility of the system modus operandi, on the one hand, in the sense of providing information to the user at all times, is always immediate – the user is able to display what he is doing at the moment and how the system is processing his actions. The visibility of the system modus operandi, on the other hand, in the sense of being capable of providing a reply/an output in real time, induces a perception of spontaneousness or natural impulse, reassuring that the user is developing a constant focus on the system, so as to effectively display his embodiment contours drawn non-stop on the interface. Beyond that, the user does not need any sort of operation guidelines in order to learn how the system works. The project also conducts an effects consistency, being able to successfully harmonize prediction, i.e. specific actions result in a same effect in similar circumstances , for example, when the user leans himself onto the floor, the sound element becomes gradually lower in pitch, and on the contrary, when the user stands up, the same sound element becomes gradually higher in pitch. The way in to information/ visual cognitive analysis is achieved through a fundamental universal design rule – the multimodal interaction, which aggregates all the human senses.
In the project YMYI, the most outstanding senses are the following: vision, which enables the user to display the ongoing animation of the whole picture; the hearing of programmed sounds (high-pitch and low-pitch), which provide, on their turn, a growing efficiency to the monitoring process that the user makes of himself and of the sonorous events that are continuously being generated in the interface. Such sounds have a tendency to react to the user movements throughout the space and as such they can function both as meaningful analysis cues and attention focuses for the user to interact with the system. The sounds are also quite effective in deepening the emotional insight/experience of the user´s interaction; the haptic-kinestesic sense, which boosts the whole implicit input towards the system, allowing us to say that the project deals with the emergent areas related to perceptual interfaces, shaped by the use of computer vision techniques and motion tracking technologies. Quoting Ben Schneiderman ( 1998):
“When an interactive system is well designed, the interface almost disappears, enabling users to concentrate on their work, exploration, or pleasure. Creating an environment in which tasks are carried out almost effortlessly…”.
We truly believe that this definition applies itself on the whole to the YMYI project. YMYI is successful in putting together both the interaction paradigm directed to the user graphic interface and the interaction paradigm directed to natural means of communication, namely by using human gesture features, which definitely enlighten the physical interactive experience.
Videos of YMYI
22nd of February, 2008
Information Systems Department
University of Minho, Portugal
From the 29th of March to the 12th April, 2008
Forum da Maia
Main room at Fórum da Maia, YMYI exhibition
Maia town, Portugal
From the 18th of April to the 20th April, 2008
SIARC - International Creative Arts Fair
Mercado Ferreira Borges
Oporto town, Portugal
From the 18th to the 20th June, 2008
International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging
YMYI Exhibition & Performance
04th July, 2008 (21:30)
Centro Cultural Vila Flôr (grande auditório)
Theater piece: "O Donzel do Cavaleiro Dom Quixote"
Written by Luis Matilla and directed by Moncho Rodriguez.
YMYI video sequences were used during the theater piece, inside an interactive stage.
10th of September, 2008
National Television Program Em Reportagem: "ninguém vive sosinho"
YMYI video sequences were used in a Video Report, reporting today's young children living without a family.
From the 15th to the 17th October, 2008 (article presentation)
3ª Conferência Nacional em Interacção Pessoa-Máquina
Article: "YMYI - You Move You Interact : Developing a dancing performance system for full body
Universidade de Évora
17th of October, 2008
Talk at Shift'08
Social and Human Ideas For Technology
Lisbon's International Fair Meeting Centre
Parque das Nações, Lisbon, Portugal
From the 06th to the 07th November, 2008 (article presentation)
ZON Digital Games Conference
Article: "You Move You Interact: Compreender os novos paradigmas de apreensão estética através de jogos digitais interactivos incorporando estádios cognitivos de aprendizagem"
Portuguese Catholic University
From the 07th to the 08th November, 2008 (article presentation)
4th International Conference on Digital Arts
Article: "You Move You Interact: a full-body dance in-between reality and virtuality"
Portuguese Catholic University
From the 10th to the 13th November, 2008 (article presentation)
II Simpósio Nacional da ABCIBER
PUC-SP - ASSOCIAÇÃO BRASILEIRA DE PESQUISADORES EM CIBECULTURA
Article: "YMYI - Uma dança virtual de corpos em metamorfose artística num ciberespaço"
Pontifícia Universidade Católica de São Paulo (PUC-SP)
S. Paulo, Brasil
João Martinho Moura; Jorge Sousa; Pedro Branco & Adérito Fernandes Marcos. University of Minho. You Move You Interact: a full-body dance in-between reality and virtuality. Proceedings of the 4th International Conference on Digital Arts - ARTECH 2008, Porto, Portugal, November 2008. Pages 49-54. ISBN 978-989-95776-3-3
João Martinho Moura; Jorge Sousa; Nelson Zagalo. University of Minho. YMYI: Compreender os novos paradigmas de apreensão estética através de jogos digitais interactivos incorporando estádios cognitivos de aprendizagem. Actas da ZON Digital Games, Porto, Portugal, November 2008. ISBN 978-989-95500-2-5
João Martinho Moura; Jorge Sousa; Pedro Branco & Adérito Fernandes Marcos. University of Minho. You Move You Interact - Developing a dancing performance system for full body interaction. Actas da 3ª Conferência Nacional em Interacção Pessoa-Máquina, Évora, Portugal, October 2008. Pag. 41-44. ISBN 972-98464-9-9
João Martinho Moura; Jorge Sousa. University of Minho. YMYI – You move you interact: uma dança virtual de corpos em metamorfose artística num ciberespaço. Actas do II Simpósio Nacional da ABCIBER (Comissão de Arte Digital), S. Paulo, Brasil, Pontifícia Universidade Católica de São Paulo (PUC-SP). November 2008
João Martinho Moura, Jorge Sousa. University of Minho. Computational Aesthetics ’08: Arts Programme. Curated Exhibition. The Eurographics Association 2008. June 2008. Pag. 144. ISBN 978-3-905674-08-8
To foster year-round engagement and dialogue within the digital, electronic, computational and media arts. Facilitate dynamic scholarship and creative programming within the ACM SIGGRAPH organization. Promote collaboration between artists and the larger computer graphics and interactive techniques community.
ACM SIGGRAPH Digital Arts Community Committee:
Victoria Szabo, Chair
Andres Burbano, Sue Gollifer, Kathy Rae Huffman, Bonnie Mitchell, Hye Yeon Nam, Derick Ostrenko, Melentie Pandilovski, Jan Searleman, Ruth West.