Improvisational Environments


The main scientific goal of this work is to gain insight into how a group of people can collaboratively make sense of their situation affectively, corporeally and intersubjectively as well as via reason and abductive — hypothesis-forming — logics in a physical environment.  Our condition is that we use all modalities available to people in physical co-presence: spatial awareness, inter-corporeal awareness, proprioception, touch balance pressure, verbal cues, gesture.

In order to experimentally vary the conditions precisely and reproducibly, we augment the physical environment with a set of software frameworks for modulating fields of spatialized sound, structured light (e.g. video or lighting), motorized or electronically augmented objects, as well as all methods of  technologies for live event.   We adapt professional techniques as well as invent custom methods when warranted.

Our design methods include:

• Minimax: maximum experiential impact for minimum engineering.

• Achieve experiential effect applying: (1) analog (the bar for rich experience and robust engineering), (2) electrical, (3) custom software, (4) custom hardware, in that order.

• No hard distinction between spectator and performer, or between patient or agent of the event.

Composing ecosystemically in responsive environments

TEI 2019 responsive environments workshop  17 March 2019, with AME faculty and researchers at Synthesis, iStage, ASU.  We workshopped how to Composing ecosystemically in responsive environments with gestural media, objects and textures.   With:

  • Jessica Rajko, movement exercises, somatic experience
  • Lauren Hayes, vibro-tactile interfaces, sonic instruments
  • Byron Lahey, “aerial” three-string robot
  • Lyu Yanjun, Connectivity Cafe (thanks to Assegid Kidane, Tejaswi Gowda, Christy Spackman, Sha Xin Wei)
  • Sha Xin Wei, Josh Gigantino, haptic vision (for phenomenological experiment with David Morris). Thanks to Assegid Kidane, Connor Rawls
  • Robert LiKamWa, haptic VR interface simulating water kinematics
  • Brandon Mechtley, Sha Xin Wei, Connor Rawls, collective, embodied, enaction of rescaled environments using responsive compositional techniques, introducing the “sc” responsive media kit + iStage blackbox environment for collective, embodied, enaction of improvisational environments using continuous ecosystem composition techniques.  Thanks to: Todd Ingalls, Pete Weisman.
  • Chris Ziegler, forest3 (  Thanks to: Connor Rawls, Pete Weisman, Chris Zlaket, Ian Shelansky.

Cloud Nursery, Connor Rawls, Pete Weisman (design by Thierry Dumont, Nima Navab, Topological Media Lab for Synthesis Atmospheres Workshop 2018)

Improvisation Workshop 2014

(1) temporal texture via relational (not ego-centered) productions of correlated patterns of dynamics and change, and (2) transitions in continuous, multivalent states and embodied agency.

Art production is a means but not the end of this Synthesis residency.  By art we mean the poetic — and poetically precise — conditioning of experience.

Scientific goals:

  1. Write up each day, each week, and at the end of the residency,  insights into, or refinements of the Core Motivating Questions (below) in your own terms.
  2. By the end of the residency, design experiments that Synthesis Center might co-host in Fall 2014 or Spring 2015 in ASU or elsewhere.

The practical goals:

  1. Bring together some ensembles* that have advanced techniques to share with other experts
  2.  Carry out an experiment in improvisation using your apparatus while resident in the AME iStage blackbox which is being renovated.

The strategic goal and value for the Synthesis Center and the School of Arts, Media + Engineering is to host the building of an apparatus, which means not only equipment and software, but also people — students + faculty + technical staff — knowing how to keep using it in creation research beyond the workshop.   (Participants from the Topological Media Lab will bear the main  responsibility for leaving behind a working apparatus as a sibling to the Ozone responsive environments apparatus at Concordia, in order to facilitate subsequent research collaborations.)Experts in the art and science of responsive environments will teach each other how to use some of the essential parts of our systems.

Residency Summary

Core Motivating Questions

• How can people moving in concert get a sense of each other’s presence and dynamics using rhythm alone?
• How can a conventionally unidimensional rhythm be extended to a temporal texture.
• How can people moving in concert anticipate each other’s dynamics without superficial representations via sound­ or light­ image?
• How can correlation be an indicator (not certificate) of collective intention?

**For more information, visit the residency website.

**For broadsheet, click here.


Sha Xin Wei: Convener | Director of Synthesis & AME

Chris Ziegler: Dance, Media & Architecture | Assist. Professor, AME

Katie Jung: Workshop Coordinator


Adrian Freed: Gesture, Ensemble, Innovation & Cultural Dynamics | Berkeley & TML Concordia

John MacCallum:  Composition, Electroacoustic Music & Percussion | Berkeley

Navid Navab: Electroacoustic Performance & Sound Art | TML Concordia

Michael Montanaro: Choreography, Movement Art & Installation | TML Concordia

Local AME Faculty

Grisha Coleman: Somatics, Movement Art & Dance | Assist. Professor, AME

Todd Ingalls: Sound Art & Installation | Synthesis & Assist. Professor, AME

Loren Olson: Real-time Video, Software Systems | Assist. Professor, AME

Garth Paine: Sound Art, Electroacoustic Music, Composition, Flute & Acoustic Ecology | Assist. Professor, AME

Pavan Turaga: Mathematics, Feature Analysis & Electrical Engineering | Assist. Professor, AME

Topological Media Lab (TML)

Julian Stein: System Integration, Response Kits (Sound, Light and Rhythm) | Synthesis, Former TML

Evan Montpellier: Real-time Video Kit & Video Instruments | TML

Research Student Participants

Byron Lahey | PhD, AME

Chinmay Dharmadhikari | MA, EE & AME

Matthew Ragan

Michael Krzyzaniak | PhD, AME

Nicole Williams

Qiao Wang | PhD, EE & AME

Alex Abreu

Cooper Sanghyun Yoo | PhD, AME, Center for Science and Imagination

Connor Rawls | BA, AME


Doug van Nort | PhD, TML, Concordia

Nikolaos Chandolias | MA, TML, Concordia

Omar Faleh | MA, TML, Concordia

Liza Solomonova | PhD TML + UdMonreal

Mayra Morales | PhD Humanities, Concordia

Bruno Zamborlin | PhD, Arts and Computer Science

Harry Smoak | TML, Concordia