INTERACTION

Software used: Pure Data, Wekinator, and Manos OSC (Leap Motion).

This project is a proof of concept for a virtual orchestra - an interactive application where audio files of singular instruments are controlled by hand movements, similarly as the movements of a conductor while working with a real ensemble. Here I used four separated tracks: bass, guitar, drums and vocal. The bass part was playing continuously as a base track, while the volume of other three instruments was controlled by hand movements. The drums were controlled by the relative distance between both hands on X axis, the guitar on Z axis, and vocal on both, X and Z axes. The project was presented at Kallio Stage in Helsinki (Finland) in December 2017.

Software used: Processing, Pure Data, Wekinator, and Manos OSC (Leap Motion).

The idea of this project is based on symbolic gestures used in a typical conversation between two people. Instead of a second person as an interlocutor, here we have the computer replying on previously defined and recorded gestures. It is not a typical dialogue between the man and a machine, since only the computer "talks": first, it asks a question in a text form (Processing), then the man gives a gesticular answer and the gesture is recognized by the computer (sonic output - Pure Data). The project was presented at Kallio Stage in Helsinki in December 2017.

The conversation:

Computer: I'm waiting...

Me: "Hello" gesture

Computer: Hello. How are you?

Me: "Bad" gesture

Computer: Maybe you could play some music? De-spa-cito...

Me: "No" gesture

Computer: OK OK...Let us hear something you like.

Me: "Perfect" gesture

Computer: Good! Let us hear it!