View source for Stimmung: Interactive sculptural installation in 3D

== Watch/Listen to the Piece Here ==

https://youtu.be/ixbs4mNZFIc

== Introduction ==

''Voices of Silicon Valley'' is a small choir dedicated to perform contemporary music in the bay area. They will be performing a piece by Karlheinz Stockhausen called Stimmung in which Megan Jurek and I are involved as handling the technical aspects of the performance. The piece itself requires 6 singers arranged in a circle in the center of a hall who sing into their microphones; these signals are then routed to an outer ring of speakers in such a way to allow the listeners to feel themselves to be in between the singers. At the time of composition very few tools were available to spatialize this signals in the outer ring; however, nowadays we count with much more technological possibilities and a lot of processing power to take this experience to a new level and do some interesting processes with it.

The piece itself will be presented twice: once in the St. Elizabeth Catholic Church in San Francisco and once in the Tateuchi Hall in Mountain View. A mix of this presentations will be done in ambisonics format and later presented in an installation composed by a 3d printed sculpture and a Binaural render of the ambisonics mix. The installation itself will allow the listener to interact with the sound-field by changing its position in the ensamble


== Regarding the Piece ==

Stimmung was composed in 1968 by Karlheinz Stockhausen as a ground breaking piece exploring new spatial possibilities as well as novelties in timber and intonation. The piece is composed around the harmonic series and the possibility to excite and play with the harmonics produced by the voice to create unique textures in just intonation. Additionally tries to reverse the dynamics of the listener with the performer trying to place, acoustically, the listener in between the performers. The proposed setup directly from Stimmung looks like this:

Additionally, the ''Voices of Silicon Valley'' group are planning to incorporate more than the proposed number of singers including 6 more singers to double each part. The piece itself, although revolutionary in its time, has a very simplistic approach to spatialization by translating the actual sound source to another location routing the source to a speaker in a one-to-one setup. However, additional techniques could be implemented during the performance to enhance the illusion of being immersed in the middle of the singers. Additionally, artificial reverb and other mixing techniques could be applied to ensure the most articulated and artistic space recreation.

== Requirements ==




== Post production ==


After the recording has been made it was mixed with Reaper to first order ambisonics with ATK plugins. Now this 4 channel audio file is fed to a C++ program developed in STK (\textit{Synthesis Toolkit}) that reads the four channels, decodes the signal to virtual speakers and then convolves each virtual speaker with a set of HRTFs to create a Binaural rendering of the mix.
Given that spatial transformations are fairly easy once the ambisonics render is done I would like to attach a knob to a small sculpture so that the rotation of the sculpture controls the rotation of the sound field. The effect of this would be a more vivid experience of a recording of a piece that was intended, since its conception, to be lived in a very specific sound field that cannot be capture in any other way.

== Technical requirements ==

Once the recording is done, most of the project will be constrained to be able to generate the ambisonics rendering which will be generated in Reaper as a DAW with ATK. The mixing process will create a 1st order ambisonics group of files that will represent what happened during the performance through a mixture of direct signals from the microphones and a general 1st order ambisonics capture in the middle of the singers. However, the real problem with translating this static set of signals into a dynamic installation will be to create an intermediate transformation matrix that will allow for the static recording to rotate at will following a potentiometer. This will require specific solutions to different problems.

'''Serial control with potentiometer'''
One requirement of the project will be a way to communicate with the system through a knob. In order to be able to do this, an arduino will translate voltage values into Serial messages and feed that information to a Raspberry Pi that will be running the reproduction system.
'''Software for matrix transformation'''
After the render is done, the 4 files will be reproduced in a C++ based software developed in \textit{STK}. This will allow an easy and controllable way to create the rotations with a self-made decoder for ambisonics management. For the serial transmission a library called \textit{termios} was used to do the management of the serial port.
'''Binaural decoding'''
The binaural decoder is going to create virtual loudspeakers in different positions in order to map an specific standard ambisonics decoder. Once the decoder has calculated the correct output signals for each of the speakers, those signals will be convolved with static impulse responses coming from a regular HRTF library (MIT Kemar library).
'''Sculpture'''
Although this part of the project itself is not directly related to the audio processing in sound, it will be an essential part of the installation. For this, I am 3D printing a sculpture to represent the piece and its properties. Accordingly I've chosen a 3D form similar to spherical harmonics that has in its longitudinal axis superposed fifths in just intonation, and in the latitudinal axis superposed fifths in equal temperament tuning system. This shape represents the conflict between modern equal temperament and just intonation and how one can preserve certain symmetry around the sphere while the other cant. \\
The equation for this shape is

	r = 20 + cos\left(\frac{3}{2} K \theta \right) + cos \left(\frac{9}{4} K \theta \right) + cos\left(2^{\frac{7}{12}} K \phi \right) + cos\left(2^{\frac{14}{12}} K \phi \right) \\
	k = 8; \quad \theta \in [0, 2 \pi]; \quad \phi \in [0, \pi];


== Final Implementation ==

At the end, the project was implemented in a Raspberry Pi with an Arduino as the interface for the controllers. Everything was assembled in a metal box which at the end only needs power and a pair of headphones making it ideal for an exhibit.\\
This is the final version of the box without the attached 3D printed structure.

The final code for all the project can be found here \\
https://github.com/jdsierral/222-FinalProject



== References ==


- Cobrum, Joe. (2016, July 11). \textit{How to make a MIDI controller with an Arduino} retrieved from: http://www.makeuseof.com/tag/make-midi-controller-arduino/
- Kronlachner, Mathias. (2014, Jan 1) \textit{Ambix v0.2.7 - Ambisonic Plug-in suite} retrieved from: http://www.matthiaskronlachner.com/?p=2015
- The ATK Community. \textit{The Ambisonic Toolkit} retrieved from: http://www.ambisonictoolkit.net
- Colafrancesco, Julien. Guillot, Pierre. Paris, Eliott. \textit{HOALibrary - High Order Ambisonics Library} retrieved from: http://www.mshparisnord.fr/hoalibrary/en/
- Greekgoddj. \textit{Ambisonic-lib - Ambisonic C++ Library} retrieved from: https://github.com/greekgoddj/ambisonic-lib
- the CIPIC HRTF Database retrieved from: http://interface.cipic.ucdavis.edu/sound/hrtf.html
- Wozniak, Tomasz. \textit{Binaural Panner - VST plugin} Retrieved from: https://codeandsound.wordpress.com/tag/hrtf/