The Fluxations Human Body Movement Interface for Comprovisational Computer Music, in collaboration with Sofia Paraskeva, is interactive algorithmic music and graphics generated spontaneously through motion detected by an infrared video camera and custom sensor gloves.

Montreal Comprovisation No.1:    (In Safari, click on the images to play videos. Or in any browser click below to load in a new window.)

Joshua B. Mailman demonstrating Fluxations in Montreal, July 15, 2012.
Click here to view the above video in a new window.

• Music: Cognition, Technology, Society, Cornell University, May 11-13, 2012

Cornell University, May 11, 2012. Photo by Evan Cortens.

• Skin-Surface-Circuit: Embodying the Improvisatory, ICASP-McGill Center for the Critical Study of Improvisation Interdisciplinary Conference, Montreal, June 14-16, 2012
Montreal Comprovisation No.2:    (In Safari, click on the images to play videos. Or in any browser click below to load in a new window.)

Joshua B. Mailman demonstrating Fluxations in Montreal, July 15, 2012.
Click here to view the above video in a new window.

The first publically presented demo of Fluxations occured on March 31, 2012 at the

• Music, Mind, and Invention Workshop (MMI) at the College of New Jersey. Click here to view the program and proceedings.

and

• Music and the Moving-Thinking Body Conference at Chancellors Hall, Senate House, University of London, April 19, 2012

Chancellors Hall, Senate House, University of London, April 19, 2012. Video by Osvaldo Glieca.

• CultureHub, Volumetric Society: Media[Circus] at LaMama, NYC, November 23-25, 2012
(In Safari, click on the images to play videos. Or in any browser click below to load in a new window.)

Sofia Paraskeva demonstrating Fluxations at LaMama, November 24, 2012.
Click here to view the above video in a new window.


Sofia Paraskeva demonstrating Fluxations at LaMama, November 24, 2012.
Click here to view the above video in a new window.


Sofia Paraskeva demonstrating Fluxations at LaMama, November 24, 2012.
Click here to view the above video in a new window.

The music algorithm is programmed by Joshua Mailman in the RTcmix language, running inside a MaxMSP patch. The graphics he programmed in Processing.

Various interactive music control moves:

Videos on vimeo document an earlier stage of development.

More detailed videos of this are on Vimeo, some with explanatory captions.

In our work The Sound and Touch of Ether's Flux we exploit new kinds of interactivity to address the relation of complexity to perception. Some interactive technologies create an immersive enviroment whose processes are influenced or triggered by participants, but in a fashion whose unpredictability pushes expression beyond reach. Other interactive music technologies do act as expressive instruments but in a non-immersive way: by either generating individual sounds directly in response to discrete gestures or by using gestures to manipulate playback of sampled sounds.

The Fluxations paradigm differs fundamentally. Our Fluxations technology allows the dancer-improviser to steer the continuing flow of spontaneously generated algorithmic music that is partly stochastic (determined by statistically controlled randomness). Thus the motion-based interactivity manipulates emergent properties (macro-properties) through continuous input from analog sensors. Continuous quantities from the analog sensors feed into the music generating algorithms to serve as statistical bounds and seeds that correlate indirectly put palpably with emergent qualities whose flux can be heard. The stochastic aspects of the algorithm create detail that is complex but yet neutral (statistically uniform) in relation to the emergent properties whose flux is expressively manipulated by the body motions of the dancer-improvisor. Thus it is observed how the user-improvisor willfully sculpts the characteristics of complexity in real-time, exemplifying a cybernetics of cybernetics (2nd order cybernetics).

The result is an algorithmically generated non-repeating stream of music that is expressive in that it enables mood shifts by allowing nuanced but audible emergent qualities to be spontaneously steered through continuous motions of the body. This approach enables a new kind of synergy between spontaneous movement, computation, and sound, thus initiating a technology-fueled fusion of dance and music, which in turn will prompt new cross-fertilizations between choreographic and sonic composition and improvisation. Because our interfaces steer fluctuating emergent qualities through continuous motion in space, they suggest tantalizing syntheses between organisms in motion and the continuous flux of their environment.

We thank the Columbia University Computer Music Center and Harvestworks for use of their space to develop this technology. Thanks also to Raphael Peterson, who programmed one of the software routines and assisted with various production issues.

Publications discussing this work:
"Improvising Synesthesia: Comprovisation of Generative Graphics and Music" in the Leonardo Electronic Almanac v.19/3, 2013.
and
“Continuous Movement, Fluid Music, and Expressive Immersive Interactive Technology: The Sound and Touch of Ether’s Flux” (co-authored with Sofia Paraskeva) in Sound, Music and the Moving-Thinking Body (edited by Osvaldo Glieca and Marilyn Wyers) Cambridge Scholars Publishing, 2013.

Stay tuned for details on more forthcoming publications documenting this work.

See also FluxNoisations, so far demonstrated at the composers forum at UC San Diego, at NoiseFloor at Staffordshire University, UK, on May 2, 2013, and at UC Santa Barbara in May and July 2013.

Return to Joshua Banks Mailman's website
Go to Sofia Paraskeva's website

© Joshua Banks Mailman and Sofia Pareskeva 2012. All rights reserved.
The documents and media here are the intellectual property of Joshua Banks Mailman and Sofia Pareskeva.