Latest revision as of 11:01, 29 November 2010

This page is intended to supplement the lecture material found in the class - providing extra tutorials, support, references for further reading, or demonstration code snippets for those interested in a given topic. Please contribute to this growing list of resources. Do you have a great explanation of how a technique works? Found a great Java applet that illustrates a concept? Discovered a great survey of the field for a particular area? Please add it for the benefit of future students. Thanks!

I encourage you to ADD links and sections - but please do not REMOVE headings or items from the page.

Timing and Segmentation

Onset Detection

Papers:
Code:

Beat Extraction

Online available beat tracking systems (compiled by Olmo Cornelis, University College Ghent, October 2010, posted on the MUSIC-IR list):

Queen Mary ‘s sonic visualiser:
http://www.sonicvisualiser.org/
with vamp plugin set:
http://vamp-plugins.org/download.html
http://www.vamp-plugins.org/download.html

Simon Dixon's BeatRoot (Java) 
http://www.eecs.qmul.ac.uk/~simond/beatroot/ 

Dan Ellis' algorithm:
http://labrosa.ee.columbia.edu/projects/coversongs/
D. Ellis (2007) Beat Tracking by Dynamic Programming, Journal of New Music Research, Special Issue on Beat and Tempo Extraction, vol. 36 no. 1, March 2007, pp. 51-60. (10pp)

Fabien Gouyon’s IBT in MARSYAS 
marsyas.info
(2 versions, 1 causal (real time) and 1 non causal)

Matthew Davies' beat-tracking (in SuperCollider by Nick Collins as "BeatTrack")
http://tinyurl.com/344ubus 

Adam Stark and Matthew Davies’ real-time beat tracker (as a Max/MSP external):
http://www.elec.qmul.ac.uk/digitalmusic/people/adams/bsa/
http://138.37.35.209/digitalmusic/papers/2009/StarkDaviesPlumbley09-dafx.pdf

Andrew Robertson has published "B-Keeper" drum tracker in Max/MSP: 
http://b-keeper.org/

Paul Brossier's Aubio C/C++ library (tempo tracking, onset detection):
http://aubio.org/

Not available online: 
- Klapuri et al. [2006] - Analysis of the Meter of Acoustic Musical Signals  
- Hainsworth [2004] - Techniques for the Automated Analysis of Musical Audio (PhD Thesis)
- Goto [2001] - An audio-based real-time beat tracking system for music with or without drum-sounds
- Schierer [1998] - Tempo and beat analysis of acoustic musical signals

Some online tools found by browsing:
http://www.zplane.de/index.php?page=home
http://www.beatunes.com/index.html
http://www.abyssmedia.com/bpmcounter/
http://users.compaqnet.be/CubicCarrot/index.htm
http://www.mp3doctor.com/free_bpm_detect.html
http://www.mixmeister.com/bpmanalyzer/bpmanalyzer.asp
http://www.softjock.com/bpmproscan.html
http://www.potionfactory.com/tangerine

Tempo Extraction

Papers:

Feature Extraction

Low Level Features

Zero Crossing, Temporal centroid, Log Attack time, Attack slope), Spectral features (Centroid, Flux, RMS, Rolloff, Flatness, Kurtosis, Brightness),Spectral bands, Log spectrogram
Chroma bins
MFCC
- Auditory Toolbox (code and docs)
MPEG-7

Higher-level features

Key Estimation
Chord Estimation
Genre (genre, artist ID, similarity)
"Fingerprints"

Visualizing and Sonifying Feature data

Matt Hoffman's feature sonification work

Analysis / Decision Making

Classification

Heuristic Analysis
Distance measures (Euclidean, Manhattan, etc.)
k-NN
SVM / One-class SVM
- Resources:
- Code:
  - libSVM (standalone, matlab, c, etc)
  - libSVM tools

Clustering and probability density models

Density distance measures (centroid distance, EMD, KL-divergence, etc)
k-Means
Clustering Demo

Clustering

GMM

HMM

There are a couple of MATLAB toolboxes for implementing HMMs and Bayesian nets, both by Kevin Murphy:

Nested classifier / Anchor-space / template-based systems

?

Model / Data Preparation Techniques

Data Preparation
- PCA / LDA
- Scaling data
Model organization
- concept, design, data set construction and organization

Evaluation Methodology

Cross Validation
Information Retrieval metrics (precision, recall, F-Measure)

Real-world applications

Audio Segmentation

Automatic Audio Segmentation: Segment Boundary and Structure Detection in Popular Music

Audio Fingerprinting

P. Cano, E. Batlle, T. Kalker, and J. Haitsma, “A review of algorithms for audio fingerprinting,” in IEEE International Workshop on Multimedia Signal Processing (MMSP), pp. 169 – 173, December 2002. 4 pages.
"On the comparison of audio fingerprints for extracting quality parameters of compressed audio"
Finding Structure in Audio for Music Information Retrieval
"Computer Vision for Music Identification" Y. Ke, D. Hoiem, and R. Sukthankar

The Last.fm fingerprinter uses this approach, code can be checked out from: svn://svn.audioscrobbler.net/recommendation/MusicID/lastfm_fplib

DAn Ellis' Matlab implementation based on Avery Wang's Shazam algorithm which works out to be pretty robust (in limited testing)

Drum Transcription

DrumTracker

Audio Similarity

A. Berenzweig, B. Logan, D. Ellis, and B. Whitman. A large-scale evaluation of acoustic and subjective music similarity measures. In Proceedings of 4th International Symposium on Music Information Retrieval, Baltimore, Maryland, 2003.

Music Recommendation / Playlisting

Getting Involved in the MIR Community

Research Databases / Collections of Ground truth data and copyright-cleared music

General MIR Datasets

http://grh.mur.at/misc/mir_datasets.html

Download links for the ISMIR 2004 genre classification contest training set:

Tags:

More:

OLPC Sound Sample Archive (8.5 GB) [1]
RWC Music Database (n DVDs) [available in Stanford Music library]
RWC - Sound Instruments Table of Contents
http://staff.aist.go.jp/m.goto/RWC-MDB/rwc-mdb-i.html
Univ or Iowa Music Instrument Samples

From Georg Holzmann: 
LIST OF PUBLIC AVAILABLE MIR DATASETS
Downloadable Datasets:
- University of Iowa musical instruments samples:
   http://theremin.music.uiowa.edu/MIS.html
   Instrument samples recorded by the University of Iowa

- ISMIR2004 Audio Description Contest Dataset:
   http://ismir2004.ismir.net/ISMIR_Contest.html
   Datasets for
   - Genre Classification/Artist Identification
   - Melody Extraction
   - Tempo Induction
   - Rhythm Classification

- Graham's Melody Extraction Dataset:
   http://www.ee.columbia.edu/~graham/mirex_melody/
   http://labrosa.ee.columbia.edu/projects/melody/
   Audio files with correspondig pitch data

- MIREX06 Audio Tempo Extraction and Beat Tracking Datasets:
 
http://www.music-ir.org/mirex/2006/index.php/Audio_Tempo_Extraction#Practice_Data

- QBSH: A Corpus for Designing QBSH (Query by Singing/Humming) Systems
   http://neural.cs.nthu.edu.tw/jang2/dataSet/childSong4public/QBSH-corpus/

- Uni Dortmund Music Audio Benchmark Data Set:
   http://www-ai.cs.uni-dortmund.de/audio.html
   Songs from different genres and with tags (from garageband.com)

- Latin Music Database:
   http://www.ppgia.pucpr.br/~silla/lmd/
   3.160 music pieces in MP3 Format classified in 10 diferent musical genres
   (only features online)


Orderable Datasets:
- RWC Music Database:
   http://staff.aist.go.jp/m.goto/RWC-MDB/
   (many CDs)
   Datasets for
   - Pop Music & Royalty-Free Music
   - Classical Music
   - Jazz Music
   - Music Genre
   - Musical Instrument Sound

   Additional: AIST RWC Annotations
   http://staff.aist.go.jp/m.goto/RWC-MDB/AIST-Annotation/
   Additional annotations to the RWC database (beat, melody, ...)

- McGill University Master Samples:
   http://www.music.mcgill.ca/resources/mums/html/
   3 DVDs with instrument samples

- USPOP2002 Pop Music data set:
   http://labrosa.ee.columbia.edu/projects/musicsim/uspop2002.html
   (3 DVDs)
   MFCC features from 706 albums and 8764 tracks (400 artists)
   with style tags

- ENST-Drums:
   http://perso.telecom-paristech.fr/~gillet/ENST-drums/
   An extensive audio-visual database for drum signals processing


Free Online Music:

- magnatune.com creative commons music:
   http://magnatune.com/info/press/coverage/ccblog

- http://www.garageband.com/
   Public domain recordings

- http://epitonic.com/
   "high quality free and legal mp3 music"

- http://www.jamendo.com/
   Creative commons licensed music

- http://musicbrainz.org/
   Get music metadata

- http://www.freesound.org/
   Collaborative database of Creative Commons licensed sounds
   (not focused on songs)

Webservices:
- Networked Environment for Music Analysis:
   http://nema.lis.uiuc.edu/
   A webservices system for submitting code, running it against virtual collections
   (full use in 2010)

- MIREX DIY Framework:
   http://www.music-ir.org/mirexdiy/
   http://www.dlib.org/dlib/december06/downie/12downie.html
   (useable ?)

MIR Software and Toolboxes

Incomplete but growing list (courtesy of Joern Loviscach):
* MARSYAS
* jAudio
* Chuck
* The Sonic Visualizer/Annotator
* CLAM
* Music-to-Knowledge (M2K)
* MIRtoolbox
* MA toolbox
* Psysound
* Praat
* IPEM
* EchoNest
* libxtract
* MuBu
* Soundspotter
* timbreID
* openSMILE
* MPEG-7 XM
* MPEG-7 Audio Encoder
* MPEG-7 Audio Analyzer
* Sphinx 4 - Java-based open-source speech recognizer  http://cmusphinx.sourceforge.net/sphinx4/#capabilities

MIR Topic Areas

From Simon Dixon, Music-IR list, Dec 2008.

MIR Systems
- Content-based Querying
- Classification (genre/style/mood)
- Recommendation / playlist generation
- Fingerprinting / DRM
- Score following / Audio alignment
- Transcription / Annotation
- Tempo induction / Beat tracking
- Summarisation
- Streaming
- Text/web mining
- Optical music recognition
- Database systems / indexing / query languages

Human issues
- user interfaces, user models
- emotion, aesthetics
- perception, cognition
- social issues
- legal and ethical issues
- business issues
- methodological and philosophical issues

Data and metadata
- audio
- MIDI
- score
- text/web
- KR schemes, standards and protocols
- libraries and collections
- test sets and evaluation

Musical knowledge
- Melody and motives
- Harmony, chords and tonality
- Rhythm, beat, tempo and form
- Timbre, instrumentation and voice
- Genre, style and mood
- Performance
- Composition
- Ethnomusicology

Difference between revisions of "MIR workshop 2008 notes"