Jump to content

MIR workshop 2008: Difference between revisions

From CCRMA Wiki
← Older editNewer edit →
Jleboeuf (talk | contribs)
500 edits
Ge (talk | contribs)
2,064 edits
No edit summary
Line 16: Line 16:
== workshop outline == Introduction to Capabilities of MIR
== workshop outline == Introduction to Capabilities of MIR


Survey of the field, Real-world applications, and challenges
Survey of the field, real-world applications, MIR research, and challenges
* Current commercial applications  
* Current commercial applications  
** Music Recommender Systems
** Music Recommender Systems
Line 24: Line 24:
** DAW technologies
** DAW technologies
** Band in a box
** Band in a box
* Large University projects
* Academic MIR research projects
** MARSYAS
** CLAM
** IMIRSEL
** Ongoing work and projects at McGill / UCSD / Columbia / Princeton / Stanford / UK / beyond
 


Signal Processing Basics (if necessary)
Signal Processing Basics (if necessary)


Feature Extraction  
=== Feature Extraction ===
* Low Level Features
* Low Level Features
** ZCR
** ZCR
** Spectral Moments (Centroid, Flux, Flatness, Kurtosis)
** "Classic" Spectral features (Centroid, Flux, RMS, Rolloff)
** Spectral Moments (Centroid, Flatness, Kurtosis)
** Spectral Bands / Filters
** Spectral Bands / Filters
** MFCC (Source-filter modeling)
** MFCC (Source-filter modeling)
** LPC
** MPEG-7
** MPEG-7


Line 43: Line 50:
** Genre (genre, artist ID, similarity)
** Genre (genre, artist ID, similarity)
** "Fingerprints"
** "Fingerprints"
 
Rhythm Analysis  
=== Classification ===
* concept and design
* genre-classification
* similarity retrieval
* instrument/speaker/source identification
 
=== Rhythm Analysis ===
* Onset Detection
* Onset Detection
* Beat Detection
* Beat Detection
* Meter detection
* Meter detection


Data reduction techniques
=== Data Reduction Techniques ===
* Linear regression  
* Linear regression  
* Threshold, Adaptive Threshold
* Threshold, Adaptive Threshold
Line 56: Line 69:
* Feature Selection
* Feature Selection


Structure and Segmentation
=== Structure and Segmentation ===
* ASA 101
* ASA 101


Classification
=== Classification Algorithms ===
* k-NN
* k-NN
* SVM
* SVM
Line 65: Line 78:
* Neural Nets
* Neural Nets


Evaluation Methodology  
=== Evaluation Methodology ===
* Data set construction
* Data set construction
* Feature selection
* Feature selection
Line 71: Line 84:
* Information Retrieval metrics (precision, recall, F-Measure)  
* Information Retrieval metrics (precision, recall, F-Measure)  


Lab Exercises
=== Lab Exercises ===
* Feature extraction from audio
* Feature extraction from audio
* Instrument Identification
* Classification tasks (e.g., instrument identification)
* Real-time MIR (vowel v consonant?) with ChucK  
* Real-time MIR with ChucK/UAna
Guest lecturer from a local music information retrieval startup?
Guest lecturer from a local music information retrieval startup?
Line 90: Line 103:
== Abstract ==  
== Abstract ==  


Music Information Retrieval (MIR) is a highly-interdisciplinary field bridging the domains of digital audio signal processing and machine learning. Simply put, MIR algorithms allow a computer to “listen” and “understand” audio, such as MP3s in a personal music collection, live streaming audio, or gigabytes of sound effects.  In the same way that listeners can recognize a song’s characteristics – tempo, key, chord progressions, genre, or song structure – MIR algorithms are capable of recognizing and extracting this information.  By understanding the characteristics of an audio selection, we can perform extensive sorting, searching, music recommendation, metadata generation, and transcription of that audio.
Music Information Retrieval (MIR) is a highly-interdisciplinary field bridging the domains of digital audio signal processing, pattern recognition, software system design, and machine learning. Simply put, MIR algorithms allow a computer to “listen” and “understand or make sense of” audio data, such as MP3's in a personal music collection, live streaming audio, or gigabytes of sound effects, in an effort to close the semantic gap between high-level musical information and low-level audio data.  In the same way that listeners can recognize the characteristics of sound and music – tempo, key, chord progressions, genre, or song structure – MIR algorithms are capable of recognizing and extracting this information, enabling systems to perform extensive sorting, searching, music recommendation, metadata generation, transcription, and even aiding/generating real-time performance.


This workshop will target students, researchers, and industry audio engineers who are unfamiliar with the field of Music Information Retrieval (MIR).  We will demonstrate the myriad of exciting technologies enabled by the fusion of basic signal processing techniques with machine learning and pattern recognition.  The presentations will be applied, multimedia-rich, overview of the building blocks of modern MIR systems.  Our goal is to make highly-interdisciplinary technologies and dauntingly-complex algorithms approachable.       
This workshop will target students, researchers, and industry audio engineers who are unfamiliar with the field of Music Information Retrieval (MIR).  We will demonstrate the myriad of exciting technologies enabled by the fusion of basic signal processing techniques with machine learning and pattern recognition.  The presentations will be applied, multimedia-rich, overview of the building blocks of modern MIR systems.  Our goal is to make highly-interdisciplinary technologies and the understanding and usage of complex algorithms approachable.       


The workshop will consist of half-day lectures, half-day supervised lab sessions, and classroom exercises and discussions.
The workshop will consist of half-day lectures, half-day supervised lab sessions, and classroom exercises and discussions.


Labs will allow students to design basic ground-up "intelligent audio" systems, use existing MIR toolboxes, applications, and complex systems.
Labs will allow students to design basic ground-up "intelligent audio" systems, use existing MIR toolboxes, programming environments, applications, and complex systems.


Knowledge of basic digital audio principles, and familiarity with basic programming (Matlab) will be useful.  
Knowledge of basic digital audio principles, and familiarity with basic programming (Matlab, C/C++, and/or ChucK) will be useful.  
Students are highly encouraged to bring their own audio source material for course demos.
Students are highly encouraged to bring their own audio source material for course demos.




[[Category: Workshops]]
[[Category: Workshops]]

Revision as of 06:45, 12 March 2008

CCRMA Workshop: Music Information Retrieval

This is Jay and Ge's brainstorming page for this summer's MIR workshop.

logistics

  • Summer 2008
  • Instructors: Jay LeBoeuf and Ge Wang


potential workshop titles

  • Music Information Retrieval
  • Information Retrieval in the Service of Music
  • Music Information Retrieval and Applications for Computer Audio
  • Intelligent Audio Systems : A review of the foundations and applications of Semantic Audio Analysis and Music Information Retrieval

== workshop outline == Introduction to Capabilities of MIR

Survey of the field, real-world applications, MIR research, and challenges

  • Current commercial applications
    • Music Recommender Systems
    • Playlisting systems
    • DJ systems
    • Music Transcription
    • DAW technologies
    • Band in a box
  • Academic MIR research projects
    • MARSYAS
    • CLAM
    • IMIRSEL
    • Ongoing work and projects at McGill / UCSD / Columbia / Princeton / Stanford / UK / beyond


Signal Processing Basics (if necessary)

Feature Extraction

  • Low Level Features
    • ZCR
    • "Classic" Spectral features (Centroid, Flux, RMS, Rolloff)
    • Spectral Moments (Centroid, Flatness, Kurtosis)
    • Spectral Bands / Filters
    • MFCC (Source-filter modeling)
    • LPC
    • MPEG-7
  • Higher-level features
    • Chroma features
    • Key estimation
    • Chords
    • Pitch Estimation
    • Genre (genre, artist ID, similarity)
    • "Fingerprints"

Classification

  • concept and design
  • genre-classification
  • similarity retrieval
  • instrument/speaker/source identification

Rhythm Analysis

  • Onset Detection
  • Beat Detection
  • Meter detection

Data Reduction Techniques

  • Linear regression
  • Threshold, Adaptive Threshold
  • Peak Picking
  • PCA / LDA
  • Feature Selection

Structure and Segmentation

  • ASA 101

Classification Algorithms

  • k-NN
  • SVM
  • HMM
  • Neural Nets

Evaluation Methodology

  • Data set construction
  • Feature selection
  • Cross Validation
  • Information Retrieval metrics (precision, recall, F-Measure)

Lab Exercises

  • Feature extraction from audio
  • Classification tasks (e.g., instrument identification)
  • Real-time MIR with ChucK/UAna

Guest lecturer from a local music information retrieval startup?

potential software, libraries, examples

  • MATLAB
  • ChucK / UAna
  • Marsyas
  • CLAM
  • Machine Learning Libraries
  • Weka Machine Learning and Data Mining Toolbox (Standalone app / Java)
  • Netlab Pattern Recognition and Clustering Toolbox (Matlab)
  • libsvm SVM toolbox (Matlab)

Abstract

Music Information Retrieval (MIR) is a highly-interdisciplinary field bridging the domains of digital audio signal processing, pattern recognition, software system design, and machine learning. Simply put, MIR algorithms allow a computer to “listen” and “understand or make sense of” audio data, such as MP3's in a personal music collection, live streaming audio, or gigabytes of sound effects, in an effort to close the semantic gap between high-level musical information and low-level audio data. In the same way that listeners can recognize the characteristics of sound and music – tempo, key, chord progressions, genre, or song structure – MIR algorithms are capable of recognizing and extracting this information, enabling systems to perform extensive sorting, searching, music recommendation, metadata generation, transcription, and even aiding/generating real-time performance.

This workshop will target students, researchers, and industry audio engineers who are unfamiliar with the field of Music Information Retrieval (MIR). We will demonstrate the myriad of exciting technologies enabled by the fusion of basic signal processing techniques with machine learning and pattern recognition. The presentations will be applied, multimedia-rich, overview of the building blocks of modern MIR systems. Our goal is to make highly-interdisciplinary technologies and the understanding and usage of complex algorithms approachable.

The workshop will consist of half-day lectures, half-day supervised lab sessions, and classroom exercises and discussions.

Labs will allow students to design basic ground-up "intelligent audio" systems, use existing MIR toolboxes, programming environments, applications, and complex systems.

Knowledge of basic digital audio principles, and familiarity with basic programming (Matlab, C/C++, and/or ChucK) will be useful. Students are highly encouraged to bring their own audio source material for course demos.

Retrieved from "http://ccrma.stanford.edu/mediawiki/index.php?title=MIR_workshop_2008&oldid=4555"