ChucK : Language Specification > Unit Analyzers




Unit AnalyzersUnit Analyzers (UAnae) are analyis building blocks, similar in concept to unit generators. They perform analysis functions on audio signals and/or metadata input, and produce metadata analysis results as output. Unit analyzers can be linked together and with unit generators to form analysis/synthesis networks. Like unit generators, several unit analyzers may run concurrently, each dynamically controlled at different rates. Because data passed between UAnae is not necessarily audio samples, and the relationship of UAna computation to time is fundamentally different than that of UGens (e.g., UAnae might compute on blocks of samples, or on metadata), the connections between UAnae have a different meaning from the connections between UGens formed with the ChucK operator, =>. This difference is reflected in the choice of a new connection operator, the upChucK operator: =^. Another key difference between UGens and UAnae is that UAnae perform analysis (only) on demand, via the upchuck() function (see below). Some more quick facts about ChucK unit analyzers:
View
a list of ChucK's builtin unit analyzer classes
View
sample code for unit analyzers
declaringUnit analyzers (UAnae) are objects, and they need to be instantiated before they can be used. We declare unit analyzers the same way we declare UGens and other objects. // instantiate an FFT, assign reference to variable f connectingThe upChucK operator (=^) is only meaningful for unit analyzers. Similar to the behavior of the ChucK operator between UGens, using =^ to connect one UAna to another connects the analysis results of the first to the analysis input of the second. // instantiate FFT and flux objects, Note that the last UAna in any chain must be chucked to the blackhole or dac to "pull" audio samples from the adc or other unit generators upstream. It
is also possible to linearly chain many UAnae together in a single
statement. In the example below, the // Set up analysis on adc, via an FFT object, a spectral flux object, and a Very importantly, it is possible to create connection networks containing both UAane and UGens. In the example below, an FFT transforms two (added) sinusoidal inputs, one of which has reverb added. An IFFT transforms the spectrum back into the time domain, and the result is processed with a third sinusoid by a gain object before being played through the dac. (No, this example is not supposed to do anything musically interesting, only help you get a feel for the syntax. Notice that any connection through which audio samples are passed is denoted with the => operator, and the connection through which spectral data is passed (from the FFT to the IFFT) is denoted with the =^ operator. //Chain a sine into a reverb, then perform FFT, then IFFT, then apply gain, then output FFT, IFFT, and other UAnae that perform transforms between the audio domain and another domain play a special role, as illustrated above. FFT takes audio samples as input, so unit generators connect to it with the ChucK operator =>. However, it outputs analysis results in the spectral domain, so it connects to other UAnae with the upChucK operator =^. Conversely, UAnae producing spectral domain output connect to the IFFT using =^, and IFFT can connect to the dac or other UGens using =>. This syntax allows the programmer to clearly reason about the expected behavior of an analysis/synthesis network, while it hides the internal mechanics of ChucK timing and sample buffering from the programmer. Finally, just as with unit generators, it is possible to dynamically disconnect unit analyzers, using the UnChucK operator (=< or !=>). controlling (over time)In any
ChucK program, it is necessary to advance time in order to pull audio
samples through the UGen network and create sound. Additionally, it is
necessary to trigger analysis computations explicitly in order for any
analysis to be performed, and for sound synthesis that depends on
analysis results (e.g., IFFT) to be performed. To explicitly trigger
computation at a point in time, the UAna's upchuck()
member function is called. In
the example below, an FFT computation is triggered every 1024 samples. adc => FFT fft => dac; In the example above, because the
FFT size is 2048 samples, the whileloop causes a standard
"slidingwindow" FFT to be computed, where the hop size is equal to
half
a
window. However, ChucK allows you to perform analysis using
nonstandard,
dynamically set, or even multiple hop sizes with the same object. For
example, in the code below, the FFT
object fft
performs
computation every 5 seconds as triggered by shred1,
and it additionally performs computation at a variable rate as
triggered by shred2. adc => FFT fft => dac; Parameters of unit analyzers may be controlled and altered at any point in time and at any control rate. We only have to assert control at the appropriate points as we move through time, by setting various parameters of the unit analyzer. To set the a value for a parameter of a UAna, a value of the proper type should be ChucKed to the corresponding control function. // connect the input to an FFT Since the control functions are member functions of the unit analyzer, the above syntax is equilavent to calling functions. For example, the line below could alternatively be used to change the FFT window to a Hamming window, as above. fft.window(Windowing.hamming(512)); For a list of unit analyzers and their control methods, consult UAna reference. Just like unit generators, to read the current value of certain parameters of a Uana, we may call an overloaded function of the same name. Additionally, assignments can be chained together when assigning one value to multiple targets. // connect adc to FFT What if a UAna that performs analysis on a group of audio samples is upchuck()ed before its internal buffer is filled? This is possible if an FFT of size 1024 is instantiated, then upchuck()ed after only 1000 samples, for example. In this case, the empty buffer slots are treated as 0's (that is, zeropadding is applied). This same behavior will occur if the FFT object's size is increased from 1024 to 2048, and then only 1023 samples pass after this change is applied; the last sample in the new (larger) buffer will be 0. Keep in mind, then, that certain analysis computations near the beginning of time and analysis computations after certain parameters have changed will logically involve a short "transient" period. // connect adc to FFT to blackhole representing metadata: the UAnaBlobIt is great to be able to trigger analysis computations like we've been doing above, but what if you want to actually use the analysis results? Luckily, calling the upchuck() function on a UAna returns a reference to an object that stores the results of any UAna analysis, called a UanaBlob. UanaBlobs can contain an array of floats, and/or an array of complex numbers (see the next section). The meaning and formatting of the UanaBlob fields is different for each UAna subtype. FFT, for example (see specification), fills in the complex array with the spectrum and the floating point array with the magnitude spectrum. Additionally, all UanaBlobs store the time when the blob was last computed.The example below demonstrates how one might access the results of an FFT: adc => FFT fft => blackhole; Beware: whenever a UAna is upchuck()ed,
the contents of its previous UAnaBlob
are overwritten. In the following code, blob1
and blob2
refer to the same UAnaBlob.
When fft.upchuck()
is called the second time, the contents of the UAnaBlob
referred to by blob1
are overwritten. adc => FFT fft => blackhole;Also beware: if time is not advanced between subsequent upchuck()s of a UAna, any upchuck() after the first will not recompute the analysis, even if UAna parameters have been changed. After the code below, blob refers to a UAnaBlob that is the result of computing the first (size 1024) FFT. adc => FFT fft => blackhole; representing complex data: the complex and polar typesIn order to represent complex data, such as the output of an FFT, two new datatypes have been added to ChucK: complex and polar. These types are described with examples here.performing analysis in UAna networksOften, the computation of one UAna will depend on the computation results of "upstream" UAnae. For example, in the UAna network below, the spectral flux is computed using the results of an FFT.adc => FFT fft =^ Flux flux => blackhole;The flow of computation in UAna networks is set up so that every time a UAna a is upchuck()ed, each UAna whose output is connected to a's input via =^ is upchuck()ed first, passing the results to a for it to use. For example, a call to flux.upchuck() will first force fft to compute an FFT on the audio samples in its buffer, then flux will use the UanaBlob from fft to compute the spectral flux. This flow of computation is handled internally by ChucK; you should understand the flow of control, but you don't need to do fft.upchuck() explicitly. Just writing code like that below will do the trick: adc => FFT fft =^ Flux flux => blackhole;Additionally, each time a UAna upchuck()s, its results are cached until time passes. This means that a UAna will only perform its computation once for a particular point in time. adc => FFT fft =^ Flux flux => blackhole;When no upchuck() is performed on a UAna, or on UAnae that depend on it, it will not do computation. For example, in the network below, the flux is never computed. adc => FFT fft =^ Flux flux => blackhole;The combination of this "computeondemand" behavior and UAna caching means that different UAnae in a network can be upchuck()ed at various/varying control rates, with maximum efficiency. In the example below, the FFT, centroid, and flux are all computed at different rates. When the analysis times for flux and fft or centroid and fft overlap, fft is computed just once due to its internal caching. When it is an analysis time point for fft but not for flux, flux will not be computed. adc => FFT fft =^ Flux flux => blackhole;An easy way to synchronize analysis of many UAnae is to upchuck() an "agglomerator" UAna. In the example below, agglom.upchuck() triggers analysis of all upstream UAnae in the network. Because agglom is only a member of the UAna base class, it does no computation of its own. However, after agglom.upchuck(), all other UAnae will have uptodate results that are synchronized, computed, and cached so that they are available to be accessed via upchuck() on each UAna (possibly by a different shred waiting for an event see below). adc => FFT fft =^ Flux flux =^ UAna agglom => blackhole;Because of the dependency and caching behavior of upchuck()ing in UAna networks, UAna feedback loops should be used with caution. In the network below, each time c is upchuck()ed, it forces b to compute, which forces a to compute, which then recognizes that b has been traversed in this upChucK path but has not been able to complete its computation thereby recognizing a loop in the network. a then uses b's last computed UAnaBlob to perform its computation. This may or may not be desirable, so be careful. adc => UAna a =^ UAna b =^ Uana c => blackhole; Another handy UAna
for synchronizing feature extraction is the FeatureCollector. Calling
upchuck() on a FeatureCollector triggers computation of all upstream
UAnae, and it concatenates their output blob data into a feature vector
that can be used as input to a classifier, for example using smirk.
adc => FFT fft =^ Flux flux =^ FeatureCollector fc => blackhole; builtin unit analyzersChucK has a number of builtin UAna classes. These classes perform many basic transform functions (FFT, IFFT) and feature extraction methods (both spectral and timedomain features). A list of builtin ChucK unit analyzers can be found here. 


