Gesture Signal Processing and Music

Gesture and Measuring Movement

• What is a gesture? Physical. Bodily: hands, face, posture. Non-verbal. What do gestures communicate? Is a gesture more like a button or a handle?
• What is a musical gesture?

Techniques for measuring human movement

• Camera-based
  • 2D image processing
  • 3D Motion Capture using many cameras
• Goniometry: measuring joint angles with physically attached sensors
  • Laetitia Sonami's Lady's Glove: http://www.youtube.com/watch?v=HYTrNOmSRSo&feature=player_embedded
• Electromyography: measures muscle activation by measuring electrical activity
  • Pamela Z's controller (which I believe uses emg): http://www.pamelaz.com/VociMov.html
• With accelerometers attached to the body (e.g. wiimote, iphone)

Really basic physics

• Position: x
• Velocity: v = dx/dt
• Acceleration: a = dv/dt
  • Proportional to force (Newton's 2nd law of motion: F = ma )
• Jerk: j = da/dt