Interaction with sound and image without a direct physical action is possible with the aid of brain computer interfaces (BCI). With this technology, `imagined’ activity, so called motor imagery can be directly linked to sound and image and influence the perception of motion in space. Placed in an immerse audio-visual environment, such a technically mediated perceptual cycle can provide a clear experience of an intentional, but inhibited movement and its reality to the mind. This shows that human perception of agency and the emergence of reality is constructed by the brain and completed in a multimodal fashion, even if certain sensory elements are missing.
Media technology allows the simulations of a physical surrounding through a few well placed and carefully balanced sound and image elements. This functional and mostly modality-specific simulacrum of a naturally occurring phenomenon may induce the brain to enter into resonance with and to reinforce the perception of the most credible scenario through completion.
State Dependency aims at realising a neurofeedback (NFB) scenario through a research experiment with artistic techniques and scientific methods. In the feedback between the (preparatory) imagination of movement and the perception of motion, the blending of modalities becomes possible, in a way that goes beyond established functional combinations of sensory inputs.
The adaptive feedback configuration proposed is based on the premise that perception depends on both inner and outer states and is dependent on reactive behaviour. In this experimental approach, the reactive loop is modulated to extend the limits of normal perception and (re-)action cycles.
Concretely, we want to combine movement control via motor imagery and audio-visual media in a 3D setup. Driven by entropy measures, these two layers can coalesce into a coherent multi-modal immersive experience that is rooted in kinaesthetic and visual perception.
The inner sense of position and movement in space of the person gets projected or linked onto a physically present, enveloping audio-visual flow of particles. The resulting amalgated experience points toward the concept of `telekinesis’. Thus, the installation can be experienced as a participant-centric telekinetic steering of the energetic particle flow. Alternatively, it can be felt as a physical translocation inside a stream, where the sense of position and centrality gets lost, even if momentarily.
The merging of the modes of position, agency, and movement sensing lead to a new experience and awareness of interaction with — projected — reality. This pre-reflective, sub-personal access to agency and a sense of position and orientation proposes a mode of experience that explores poly-modal kinaesthetically-driven awareness, hitherto an uncharted territory.
See published article here
