256a-fall-2022/hw2

Homework #2: Sound Peeking

Due Dates

• HW2 milestone 1: due Friday (10/7, 6pm): all three initial tutorials completed; HW2 webpage with progress report on learning/playing with Chunity + 2-3 brainstorm ideas
• HW2 milestone 2: due Monday (10/17, 1pm): HW2 webpage updated with milestone video + in-class critique
• HW2 final deliverable: due Monday (10/24, 1pm) HW2 final webpage/video + in-class presentation

reminder: please also organize your weekly reading responses online at http://ccrma.stanford.edu/~YOURID/256a/rrXXX/) -- replace XXX with the number of of the reading response.

In this assignment, you are to visualize sound in real-time, using Unity for the graphics programming, and ChucK (via Chunity) as a sound source. Your program will visualize: 1) live microphone input and 2) a designed sound narrative written in ChucK. This is a fun project but there are many moving parts; start early.

Specification (part 1a of 3): Tutorials + Readings

• follow and do the Unity Roll-a-Ball tutorial in a new project
• next (in the same project), follow and do the Chunity tutorials
• download and play with sndpeek for ideas
• Artful Design chapter 2 reading response (and Aaru video viewing; see https://ccrma.stanford.edu/courses/256a/docs/reading-responses.pdf): due Sunday (10/9) 11:59pm
• Artful Design chapter 3 reading response: due Sunday (10/16) 11:59pm

Specification (part 1b of 3): Real-time Sound Input in Unity

• follow and do Ge and Kunwoo's Artful Design TV tutorial "HelloAudioVisualizer" in a new project (do Roll-a-Ball and Chunity tutorials first)
  • note: the above tutorial does not use Chunity! Use the starter code in the next step to set up your project
• start working on your own visualizer in a new project using the ChunityAudioVisualizer starter code => Unity Package | scripts only
• end up with a usable Unity project as a starting point for your project
  • basic visualization of time-domain waveform and magnitude spectrum
  • test using the microphone input!

Specification (part 2 of 3): Visualizing the Spectrum History

• design and implement a waterfall plot (like sndpeek), a real-time scrolling spectrogram, or another way to display spectrums over time
  • "make it read"!
  • feel free to experiment on how to represent the spectrum history visually

Specification (part 3 of 3): An Audio-Visual Narrative

• using Chunity, create a ChucK program to run inside your visualizer
  • can use a combination of microphone (adc in ChucK) and sound synthesis
  • think about different "sections" or "movements", and how to transition between them
  • try to align the aesthetic of your visualizer and your ChucK program (give it personality)
  • (optional): can use keyboard input
• aesthetic goal:
  • polish not important!
  • technical fanciness not important!
  • making the viewer/listener feel something: important!!

Note

• have fun with it!!!
• comment your code!
• you are welcome to work together, but you must do/turn in your own work

Final Deliverables

• 0) reading responses, to Chapters 2 and 3 (upload URL to Canvas, as per usual)
• 1) HW2 project webpage (should be viewable at http://ccrma.stanford.edu/~YOURID/256a/hw2/). It should include:
  • a name and description of your audio visualizer
  • a few screenshots that capture various parts of your visualizer in action
  • a polished high-resolution video (2-3 minutes, with audio, screen capture or with camera) of both your visualizer in action AND your narrative
    • hint: create a production build from Unity and capture video + audio using this build in FULL-SCREEN
    • upload to YouTube or Vimeo (or CCRMA) and embed the video from your webpage
  • instructions for using your visualizers, including any keyboard/mouse control
  • the production build of your visualizer (please specify the platform e.g., MacOS, Windows, or Linux)
  • your Unity project (stripped of unnecessary and intermediate files; just the essentials needed to build and run the project)
  • a final short text section that:
    • conveys your ideas/comments in constructing each program
    • describes any difficulties you encountered in the process
    • acknowledges help you received for this project
  • feel free to include (or not include) your previous HW2 milestones. If you do, these previous milestones should appear AT THE BOTTOM of your HW2 webpage.
• 2) submit to Canvas: URL to webpage and your Chunity code.