Difference between revisions of "MIR workshop 2008 notes"

From CCRMA Wiki
Jump to: navigation, search
(→‎Beat Extraction)
 
(64 intermediate revisions by 3 users not shown)
Line 4: Line 4:
 
I encourage you to ADD links and sections - but please do not REMOVE headings or items from the page.
 
I encourage you to ADD links and sections - but please do not REMOVE headings or items from the page.
  
−
== Timing and Segmentation ==  
+
= Timing and Segmentation =  
−
=== Onset Detection ===  
+
== Onset Detection ==  
−
==== Papers ====
+
* Papers:
−
==== Code ====
+
* Code:
−
=== Beat Extraction ===   
+
== Beat Extraction ==   
−
==== Papers ====
+
Online available beat tracking systems (compiled by Olmo Cornelis, University College Ghent, October 2010, posted on the MUSIC-IR list):
−
==== Code ====
+
<pre>
−
=== Tempo Extraction ===
+
Queen Mary ‘s sonic visualiser:
−
==== Papers ====
+
http://www.sonicvisualiser.org/
−
==== Code ====
+
with vamp plugin set:
 +
http://vamp-plugins.org/download.html
 +
http://www.vamp-plugins.org/download.html
  
−
== Feature Extraction ==
+
Simon Dixon's BeatRoot (Java)  
−
== Low Level Features ==
+
http://www.eecs.qmul.ac.uk/~simond/beatroot/
−
=== Zero Crossing, Temporal centroid, Log Attack time, Attack slope), Spectral features (Centroid, Flux, RMS, Rolloff, Flatness, Kurtosis, Brightness),Spectral bands, Log spectrogram ===
+
−
=== Chroma bins ===
+
−
=== MFCC ===
+
−
=== MPEG-7 ===
+
  
 +
Dan Ellis' algorithm:
 +
http://labrosa.ee.columbia.edu/projects/coversongs/
 +
D. Ellis (2007) Beat Tracking by Dynamic Programming, Journal of New Music Research, Special Issue on Beat and Tempo Extraction, vol. 36 no. 1, March 2007, pp. 51-60. (10pp)
 +
 +
Fabien Gouyon’s IBT in MARSYAS
 +
marsyas.info
 +
(2 versions, 1 causal (real time) and 1 non causal)
 +
 +
Matthew Davies' beat-tracking (in SuperCollider by Nick Collins as "BeatTrack")
 +
http://tinyurl.com/344ubus
 +
 +
Adam Stark and Matthew Davies’ real-time beat tracker (as a Max/MSP external):
 +
http://www.elec.qmul.ac.uk/digitalmusic/people/adams/bsa/
 +
http://138.37.35.209/digitalmusic/papers/2009/StarkDaviesPlumbley09-dafx.pdf
 +
 +
Andrew Robertson has published "B-Keeper" drum tracker in Max/MSP:
 +
http://b-keeper.org/
 +
 +
Paul Brossier's Aubio C/C++ library (tempo tracking, onset detection):
 +
http://aubio.org/
 +
 +
Not available online:
 +
- Klapuri et al. [2006] - Analysis of the Meter of Acoustic Musical Signals 
 +
- Hainsworth [2004] - Techniques for the Automated Analysis of Musical Audio (PhD Thesis)
 +
- Goto [2001] - An audio-based real-time beat tracking system for music with or without drum-sounds
 +
- Schierer [1998] - Tempo and beat analysis of acoustic musical signals
 +
 +
Some online tools found by browsing:
 +
http://www.zplane.de/index.php?page=home
 +
http://www.beatunes.com/index.html
 +
http://www.abyssmedia.com/bpmcounter/
 +
http://users.compaqnet.be/CubicCarrot/index.htm
 +
http://www.mp3doctor.com/free_bpm_detect.html
 +
http://www.mixmeister.com/bpmanalyzer/bpmanalyzer.asp
 +
http://www.softjock.com/bpmproscan.html
 +
http://www.potionfactory.com/tangerine
 +
</pre>
 +
 +
== Tempo Extraction ==
 +
* Papers:
 +
** [http://www.inf.ed.ac.uk/teaching/courses/inf2b/learnnotes/inf2b-learn14-notes.pdf Tempo and beat analysis of acoustic musical signals]
 +
** [http://www.wellesley.edu/Physics/brown/pubs/meterACv94P1953-P1957.pdf Determination of the meter of musical scores by autocorrelation]
 +
** [http://www.iro.umontreal.ca/~pift6080/documents/papers/scheirer_jasa.pdf Tempo and beat analysis of acoustic musical signals]
 +
** [http://staff.aist.go.jp/m.goto/PAPER/JNMR2001goto.pdf An Audio-based Real-time Beat Tracking System for Music With or Without Drum-sounds]*
 +
 +
= Feature Extraction =
 +
== Low Level Features ==
 +
* Zero Crossing, Temporal centroid, Log Attack time, Attack slope), Spectral features (Centroid, Flux, RMS, Rolloff, Flatness, Kurtosis, Brightness),Spectral bands, Log spectrogram
 +
* Chroma bins
 +
* MFCC
 +
** [http://cobweb.ecn.purdue.edu/~malcolm/interval/1998-010/ Auditory Toolbox (code and docs)]
 +
* MPEG-7
 
== Higher-level features ==  
 
== Higher-level features ==  
−
=== Key Estimation ===
+
* Key Estimation
−
=== Chord Estimation ===
+
* Chord Estimation
−
=== Genre (genre, artist ID, similarity) ===  
+
* Genre (genre, artist ID, similarity)
−
=== "Fingerprints" ===
+
* "Fingerprints"
 +
== Visualizing and Sonifying Feature data ==  
 +
* [http://www.cs.princeton.edu/~mdhoffma/icmc2008/ Matt Hoffman's feature sonification work]
 +
 
 +
= Analysis / Decision Making =
 +
== Classification ==
 +
* Heuristic Analysis
 +
* Distance measures (Euclidean, Manhattan, etc.)
 +
* k-NN
 +
* SVM / One-class SVM
 +
** Resources:
 +
*** [http://homepages.cae.wisc.edu/~ece539/matlab/ The interactive Matlab SVM Demo that I demonstrated on Lecture 5 comes from here]
 +
*** [http://www.eee.metu.edu.tr/~alatan/Courses/Demo/AppletSVM.html A nice SVM java applet to demo the concepts]
 +
*** [http://www.autonlab.org/tutorials/svm15.pdf Andrew Moore's SVM Powerpoint Lecture]
 +
*** [http://www.kernel-machines.org/ User community of SVM enthusiasts]
 +
*** [http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf A practical guide to SVM classification]
 +
*** [http://www.kyb.tuebingen.mpg.de/bs/people/weston/svmpractical/ SVM Practical (How to get good results without cheating)]
 +
*** [https://list.scms.waikato.ac.nz/pipermail/wekalist/2006-November/008533.html One-class SVM posting]
 +
** Code:
 +
*** [http://www.csie.ntu.edu.tw/~cjlin/libsvm/ libSVM (standalone, matlab, c, etc)]
 +
*** [http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/ libSVM tools]
  
−
== Analysis / Decision Making ==
 
−
== Classification ==
 
−
=== Heuristic Analysis ===
 
−
=== Distance measures (Euclidean, Manhattan, etc.) ===
 
−
=== k-NN ===
 
−
=== SVM / One-class SVM ===
 
 
==  Clustering and probability density models ==  
 
==  Clustering and probability density models ==  
−
=== Density distance measures (centroid distance, EMD, KL-divergence, etc) ===
+
* Density distance measures (centroid distance, EMD, KL-divergence, etc)  
−
=== k-Means ===
+
* k-Means  
 +
* [http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html Clustering Demo]
 +
 
 
=== Clustering ===  
 
=== Clustering ===  
−
=== GMM ===  
+
* GMM
−
=== HMM  ===  
+
** [http://www.inf.ed.ac.uk/teaching/courses/inf2b/learnnotes/inf2b-learn02-notes.pdf Simple review of probability with introduction of Bayes Rules ]
−
==  Nested classifier / Anchor-space / template-based systems ==  
+
** [http://en.wikipedia.org/wiki/Conditional_probability Good description of conditional probabilities]
 +
** [http://crow.ee.washington.edu/people/bulyko/papers/em.pdf EM explained]
 +
** [http://www.cs.cmu.edu/~alad/em/ Expectation-Maximization Java Applet]
 +
** [http://www.ee.columbia.edu/~dpwe/muscontent/ Lab featuring real-world GMM examples for singing detection]
 +
** [http://www.ee.columbia.edu/~dpwe/e6820/outline.html Dan Ellis' Speech and Audio Processing Lectures]
 +
 
 +
=== HMM  ===
 +
* [http://www.comp.leeds.ac.uk/roger/HiddenMarkovModels/html_dev/main.html High-level introduction to HMM]
 +
* [http://www.ee.columbia.edu/dpwe/e6820/papers/Rabiner89-hmm.pdf “A tutorial on hidden markov models and selected applications in speech recognition”  Lawrence Rabiner, Proc. IEEE, 77(2), Feb 1989.]
 +
* [http://www.ee.columbia.edu/dpwe/e6820/matlab/epflhmm/labman2.pdf A self-directed introduction / lab for HMMs]
 +
* [http://www.mathworks.com/access/helpdesk/help/toolbox/stats/index.html?/access/helpdesk/help/toolbox/stats/f8368.html&http://www.google.com/search?q=As+an+example%2C+consider+a+Markov+model+with+two+states+and+six+possible+emissions.+The+model+uses%3A&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a Matlab Introduction to HMM functions]
 +
 
 +
There are a couple of MATLAB toolboxes for implementing HMMs and Bayesian nets, both by Kevin Murphy:
 +
 
 +
* [http://www.cs.ubc.ca/~murphyk/Software/HMM/hmm.html HMM toolbox]
 +
* [http://code.google.com/p/bnt/ BNT]
 +
 
 +
==  Nested classifier / Anchor-space / template-based systems ==
 +
* ?
 +
 
 +
= Model / Data Preparation Techniques =
 +
* Data Preparation
 +
** PCA / LDA 
 +
** Scaling data
 +
* Model organization
 +
** concept, design, data set construction and organization
 +
 
 +
= Evaluation Methodology =
 +
* Cross Validation
 +
* Information Retrieval metrics (precision, recall, F-Measure)
 +
 
 +
= Real-world applications =
 +
== Audio Segmentation ==
 +
[http://www.ifs.tuwien.ac.at/mir/audiosegmentation.html Automatic Audio Segmentation: Segment Boundary and Structure Detection in Popular Music]
 +
 
 +
== Audio Fingerprinting ==
 +
* [http://ieeexplore.ieee.org/iel5/8561/27103/01203274.pdf?tp=&isnumber=&arnumber=1203274 P. Cano, E. Batlle, T. Kalker, and J. Haitsma, “A review of algorithms for audio fingerprinting,” in IEEE International Workshop on Multimedia Signal Processing (MMSP), pp. 169 – 173, December 2002. 4 pages.]
 +
* [http://ict.ewi.tudelft.nl/pub/doets/Paper_Doets_SPIE_January_2006.pdf "On the comparison of audio fingerprints for extracting quality parameters of compressed audio"]
 +
* [http://www.cs.northwestern.edu/~pardo/publications/pardo-IEEE-signal-processing-mag-06.pdf Finding Structure in Audio for Music Information Retrieval]
 +
* "Computer Vision for Music Identification" Y. Ke, D. Hoiem, and R. Sukthankar
 +
The Last.fm fingerprinter uses this approach, code can be checked out from: svn://svn.audioscrobbler.net/recommendation/MusicID/lastfm_fplib
 +
* [http://www.ee.columbia.edu/~dpwe/resources/matlab/fingerprint/ DAn Ellis' Matlab implementation based on Avery Wang's Shazam algorithm which works out to be pretty robust (in limited testing)]
 +
 
 +
== Drum Transcription ==
 +
* [http://www.toontrack.com/drumtracker.asp DrumTracker]
 +
== Audio Similarity ==
 +
* [http://alumni.media.mit.edu/~bwhitman/ismir03-sim.pdf A. Berenzweig, B. Logan, D. Ellis, and B. Whitman. A large-scale evaluation of acoustic and subjective music similarity measures. In Proceedings of 4th International Symposium on Music Information Retrieval, Baltimore, Maryland, 2003.]
 +
== Music Recommendation / Playlisting ==
 +
* [http://mtg.upf.edu/~ocelma/MusicRecommendationTutorial-ISMIR2007/ ISMIR 2007 Music Recommendation Tutorial]
 +
* [http://www.audioscrobbler.net/data/webservices/ Social Tag info from Last.fm]
 +
 
 +
= Getting Involved in the MIR Community =
 +
* [http://www.ismir.net/ ISMIR]
 +
* [http://www.aes.org/technical/saa/ Audio Engineering Society and the Semantic Audio Analysis Technical Committee]
 +
* [http://listes.ircam.fr/wws/info/music-ir MUSIC-IR list]
 +
= Research Databases / Collections of Ground truth data and copyright-cleared music =
 +
* [http://cosmal.ucsd.edu/cal/projects/AnnRet/AnnRet.php CAL500]
 +
* [http://staff.aist.go.jp/m.goto/RWC-MDB/ RWC ]
 +
* [http://labrosa.ee.columbia.edu/projects/musicsim/uspop2002.html USPop ]
 +
* [http://www.music-ir.org/mirex/2005/index.php/Audio_Genre_Classification Magnatunes]
 +
<br> General MIR Datasets
 +
* http://grh.mur.at/misc/mir_datasets.html
 +
<br>Download links for the ISMIR 2004 genre classification contest training set:<br>
 +
* http://ismir2004.ismir.net/genre_contest/index.htm
 +
* http://www.iua.upf.es/mtg/ismir2004/contest/Training_Tracks1.tar.gz
 +
* http://www.iua.upf.es/mtg/ismir2004/contest/Training_Tracks2.tar.gz
 +
<br>Tags:
 +
* http://SocialMusicResearch.org/data
 +
* http://SocialMusicResearch.org/code
 +
 
 +
More:
 +
* OLPC Sound Sample Archive (8.5 GB) [http://wiki.laptop.org/go/Sound_samples]
 +
* RWC Music Database (n DVDs) [available in Stanford Music library]
 +
* [http://staff.aist.go.jp/m.goto/RWC-MDB/rwc-mdb-i.html RWC - Sound Instruments Table of Contents]
 +
* http://staff.aist.go.jp/m.goto/RWC-MDB/rwc-mdb-i.html
 +
* [http://theremin.music.uiowa.edu/MIS.html Univ or Iowa Music Instrument Samples ]
 +
 
 +
 
 +
<pre>
 +
From Georg Holzmann:
 +
LIST OF PUBLIC AVAILABLE MIR DATASETS
 +
Downloadable Datasets:
 +
- University of Iowa musical instruments samples:
 +
  http://theremin.music.uiowa.edu/MIS.html
 +
  Instrument samples recorded by the University of Iowa
 +
 
 +
- ISMIR2004 Audio Description Contest Dataset:
 +
  http://ismir2004.ismir.net/ISMIR_Contest.html
 +
  Datasets for
 +
  - Genre Classification/Artist Identification
 +
  - Melody Extraction
 +
  - Tempo Induction
 +
  - Rhythm Classification
 +
 
 +
- Graham's Melody Extraction Dataset:
 +
  http://www.ee.columbia.edu/~graham/mirex_melody/
 +
  http://labrosa.ee.columbia.edu/projects/melody/
 +
  Audio files with correspondig pitch data
 +
 
 +
- MIREX06 Audio Tempo Extraction and Beat Tracking Datasets:
 +
 +
http://www.music-ir.org/mirex/2006/index.php/Audio_Tempo_Extraction#Practice_Data
 +
 
 +
- QBSH: A Corpus for Designing QBSH (Query by Singing/Humming) Systems
 +
  http://neural.cs.nthu.edu.tw/jang2/dataSet/childSong4public/QBSH-corpus/
 +
 
 +
- Uni Dortmund Music Audio Benchmark Data Set:
 +
  http://www-ai.cs.uni-dortmund.de/audio.html
 +
  Songs from different genres and with tags (from garageband.com)
 +
 
 +
- Latin Music Database:
 +
  http://www.ppgia.pucpr.br/~silla/lmd/
 +
  3.160 music pieces in MP3 Format classified in 10 diferent musical genres
 +
  (only features online)
 +
 
 +
 
 +
Orderable Datasets:
 +
- RWC Music Database:
 +
  http://staff.aist.go.jp/m.goto/RWC-MDB/
 +
  (many CDs)
 +
  Datasets for
 +
  - Pop Music & Royalty-Free Music
 +
  - Classical Music
 +
  - Jazz Music
 +
  - Music Genre
 +
  - Musical Instrument Sound
 +
 
 +
  Additional: AIST RWC Annotations
 +
  http://staff.aist.go.jp/m.goto/RWC-MDB/AIST-Annotation/
 +
  Additional annotations to the RWC database (beat, melody, ...)
 +
 
 +
- McGill University Master Samples:
 +
  http://www.music.mcgill.ca/resources/mums/html/
 +
  3 DVDs with instrument samples
 +
 
 +
- USPOP2002 Pop Music data set:
 +
  http://labrosa.ee.columbia.edu/projects/musicsim/uspop2002.html
 +
  (3 DVDs)
 +
  MFCC features from 706 albums and 8764 tracks (400 artists)
 +
  with style tags
 +
 
 +
- ENST-Drums:
 +
  http://perso.telecom-paristech.fr/~gillet/ENST-drums/
 +
  An extensive audio-visual database for drum signals processing
 +
 
 +
 
 +
Free Online Music:
 +
 
 +
- magnatune.com creative commons music:
 +
  http://magnatune.com/info/press/coverage/ccblog
 +
 
 +
- http://www.garageband.com/
 +
  Public domain recordings
 +
 
 +
- http://epitonic.com/
 +
  "high quality free and legal mp3 music"
 +
 
 +
- http://www.jamendo.com/
 +
  Creative commons licensed music
 +
 
 +
- http://musicbrainz.org/
 +
  Get music metadata
 +
 
 +
- http://www.freesound.org/
 +
  Collaborative database of Creative Commons licensed sounds
 +
  (not focused on songs)
 +
 
 +
Webservices:
 +
- Networked Environment for Music Analysis:
 +
  http://nema.lis.uiuc.edu/
 +
  A webservices system for submitting code, running it against virtual collections
 +
  (full use in 2010)
 +
 
 +
- MIREX DIY Framework:
 +
  http://www.music-ir.org/mirexdiy/
 +
  http://www.dlib.org/dlib/december06/downie/12downie.html
 +
  (useable ?)
 +
</pre>
 +
 
 +
= MIR Software and Toolboxes =
 +
<pre>
 +
Incomplete but growing list (courtesy of Joern Loviscach):
 +
* MARSYAS
 +
* jAudio
 +
* Chuck
 +
* The Sonic Visualizer/Annotator
 +
* CLAM
 +
* Music-to-Knowledge (M2K)
 +
* MIRtoolbox
 +
* MA toolbox
 +
* Psysound
 +
* Praat
 +
* IPEM
 +
* EchoNest
 +
* libxtract
 +
* MuBu
 +
* Soundspotter
 +
* timbreID
 +
* openSMILE
 +
* MPEG-7 XM
 +
* MPEG-7 Audio Encoder
 +
* MPEG-7 Audio Analyzer
 +
* Sphinx 4 - Java-based open-source speech recognizer  http://cmusphinx.sourceforge.net/sphinx4/#capabilities
 +
</pre>
 +
 
 +
= MIR Topic Areas =
 +
From Simon Dixon, Music-IR list, Dec 2008.
 +
<pre>
 +
MIR Systems
 +
- Content-based Querying
 +
- Classification (genre/style/mood)
 +
- Recommendation / playlist generation
 +
- Fingerprinting / DRM
 +
- Score following / Audio alignment
 +
- Transcription / Annotation
 +
- Tempo induction / Beat tracking
 +
- Summarisation
 +
- Streaming
 +
- Text/web mining
 +
- Optical music recognition
 +
- Database systems / indexing / query languages
 +
 
 +
Human issues
 +
- user interfaces, user models
 +
- emotion, aesthetics
 +
- perception, cognition
 +
- social issues
 +
- legal and ethical issues
 +
- business issues
 +
- methodological and philosophical issues
 +
 
 +
Data and metadata
 +
- audio
 +
- MIDI
 +
- score
 +
- text/web
 +
- KR schemes, standards and protocols
 +
- libraries and collections
 +
- test sets and evaluation
  
−
== Model / Data Preparation Techniques ==
+
Musical knowledge
 +
- Melody and motives
 +
- Harmony, chords and tonality
 +
- Rhythm, beat, tempo and form
 +
- Timbre, instrumentation and voice
 +
- Genre, style and mood
 +
- Performance
 +
- Composition
 +
- Ethnomusicology
 +
</pre>

Latest revision as of 11:01, 29 November 2010

This page is intended to supplement the lecture material found in the class - providing extra tutorials, support, references for further reading, or demonstration code snippets for those interested in a given topic. Please contribute to this growing list of resources. Do you have a great explanation of how a technique works? Found a great Java applet that illustrates a concept? Discovered a great survey of the field for a particular area? Please add it for the benefit of future students. Thanks!

I encourage you to ADD links and sections - but please do not REMOVE headings or items from the page.

Timing and Segmentation

Onset Detection

  • Papers:
  • Code:

Beat Extraction

Online available beat tracking systems (compiled by Olmo Cornelis, University College Ghent, October 2010, posted on the MUSIC-IR list): 

Queen Mary ‘s sonic visualiser:
http://www.sonicvisualiser.org/
with vamp plugin set:
http://vamp-plugins.org/download.html
http://www.vamp-plugins.org/download.html

Simon Dixon's BeatRoot (Java) 
http://www.eecs.qmul.ac.uk/~simond/beatroot/ 

Dan Ellis' algorithm:
http://labrosa.ee.columbia.edu/projects/coversongs/
D. Ellis (2007) Beat Tracking by Dynamic Programming, Journal of New Music Research, Special Issue on Beat and Tempo Extraction, vol. 36 no. 1, March 2007, pp. 51-60. (10pp)

Fabien Gouyon’s IBT in MARSYAS 
marsyas.info
(2 versions, 1 causal (real time) and 1 non causal)

Matthew Davies' beat-tracking (in SuperCollider by Nick Collins as "BeatTrack")
http://tinyurl.com/344ubus 

Adam Stark and Matthew Davies’ real-time beat tracker (as a Max/MSP external):
http://www.elec.qmul.ac.uk/digitalmusic/people/adams/bsa/
http://138.37.35.209/digitalmusic/papers/2009/StarkDaviesPlumbley09-dafx.pdf

Andrew Robertson has published "B-Keeper" drum tracker in Max/MSP: 
http://b-keeper.org/

Paul Brossier's Aubio C/C++ library (tempo tracking, onset detection):
http://aubio.org/

Not available online: 
- Klapuri et al. [2006] - Analysis of the Meter of Acoustic Musical Signals  
- Hainsworth [2004] - Techniques for the Automated Analysis of Musical Audio (PhD Thesis)
- Goto [2001] - An audio-based real-time beat tracking system for music with or without drum-sounds
- Schierer [1998] - Tempo and beat analysis of acoustic musical signals

Some online tools found by browsing:
http://www.zplane.de/index.php?page=home
http://www.beatunes.com/index.html
http://www.abyssmedia.com/bpmcounter/
http://users.compaqnet.be/CubicCarrot/index.htm
http://www.mp3doctor.com/free_bpm_detect.html
http://www.mixmeister.com/bpmanalyzer/bpmanalyzer.asp
http://www.softjock.com/bpmproscan.html
http://www.potionfactory.com/tangerine

Tempo Extraction

Feature Extraction

Low Level Features

  • Zero Crossing, Temporal centroid, Log Attack time, Attack slope), Spectral features (Centroid, Flux, RMS, Rolloff, Flatness, Kurtosis, Brightness),Spectral bands, Log spectrogram
  • Chroma bins
  • MFCC
  • MPEG-7

Higher-level features

  • Key Estimation
  • Chord Estimation
  • Genre (genre, artist ID, similarity)
  • "Fingerprints"

Visualizing and Sonifying Feature data

Analysis / Decision Making

Classification

Clustering and probability density models

  • Density distance measures (centroid distance, EMD, KL-divergence, etc)
  • k-Means
  • Clustering Demo

Clustering

HMM

There are a couple of MATLAB toolboxes for implementing HMMs and Bayesian nets, both by Kevin Murphy:

Nested classifier / Anchor-space / template-based systems

  •  ?

Model / Data Preparation Techniques

  • Data Preparation
    • PCA / LDA
    • Scaling data
  • Model organization
    • concept, design, data set construction and organization

Evaluation Methodology

  • Cross Validation
  • Information Retrieval metrics (precision, recall, F-Measure)

Real-world applications

Audio Segmentation

Automatic Audio Segmentation: Segment Boundary and Structure Detection in Popular Music

Audio Fingerprinting

The Last.fm fingerprinter uses this approach, code can be checked out from: svn://svn.audioscrobbler.net/recommendation/MusicID/lastfm_fplib

Drum Transcription

Audio Similarity

Music Recommendation / Playlisting

Getting Involved in the MIR Community

Research Databases / Collections of Ground truth data and copyright-cleared music


General MIR Datasets


Download links for the ISMIR 2004 genre classification contest training set:


Tags:

More: 


From Georg Holzmann: 
LIST OF PUBLIC AVAILABLE MIR DATASETS
Downloadable Datasets:
- University of Iowa musical instruments samples:
   http://theremin.music.uiowa.edu/MIS.html
   Instrument samples recorded by the University of Iowa

- ISMIR2004 Audio Description Contest Dataset:
   http://ismir2004.ismir.net/ISMIR_Contest.html
   Datasets for
   - Genre Classification/Artist Identification
   - Melody Extraction
   - Tempo Induction
   - Rhythm Classification

- Graham's Melody Extraction Dataset:
   http://www.ee.columbia.edu/~graham/mirex_melody/
   http://labrosa.ee.columbia.edu/projects/melody/
   Audio files with correspondig pitch data

- MIREX06 Audio Tempo Extraction and Beat Tracking Datasets:
 
http://www.music-ir.org/mirex/2006/index.php/Audio_Tempo_Extraction#Practice_Data

- QBSH: A Corpus for Designing QBSH (Query by Singing/Humming) Systems
   http://neural.cs.nthu.edu.tw/jang2/dataSet/childSong4public/QBSH-corpus/

- Uni Dortmund Music Audio Benchmark Data Set:
   http://www-ai.cs.uni-dortmund.de/audio.html
   Songs from different genres and with tags (from garageband.com)

- Latin Music Database:
   http://www.ppgia.pucpr.br/~silla/lmd/
   3.160 music pieces in MP3 Format classified in 10 diferent musical genres
   (only features online)


Orderable Datasets:
- RWC Music Database:
   http://staff.aist.go.jp/m.goto/RWC-MDB/
   (many CDs)
   Datasets for
   - Pop Music & Royalty-Free Music
   - Classical Music
   - Jazz Music
   - Music Genre
   - Musical Instrument Sound

   Additional: AIST RWC Annotations
   http://staff.aist.go.jp/m.goto/RWC-MDB/AIST-Annotation/
   Additional annotations to the RWC database (beat, melody, ...)

- McGill University Master Samples:
   http://www.music.mcgill.ca/resources/mums/html/
   3 DVDs with instrument samples

- USPOP2002 Pop Music data set:
   http://labrosa.ee.columbia.edu/projects/musicsim/uspop2002.html
   (3 DVDs)
   MFCC features from 706 albums and 8764 tracks (400 artists)
   with style tags

- ENST-Drums:
   http://perso.telecom-paristech.fr/~gillet/ENST-drums/
   An extensive audio-visual database for drum signals processing


Free Online Music:

- magnatune.com creative commons music:
   http://magnatune.com/info/press/coverage/ccblog

- http://www.garageband.com/
   Public domain recordings

- http://epitonic.com/
   "high quality free and legal mp3 music"

- http://www.jamendo.com/
   Creative commons licensed music

- http://musicbrainz.org/
   Get music metadata

- http://www.freesound.org/
   Collaborative database of Creative Commons licensed sounds
   (not focused on songs)

Webservices:
- Networked Environment for Music Analysis:
   http://nema.lis.uiuc.edu/
   A webservices system for submitting code, running it against virtual collections
   (full use in 2010)

- MIREX DIY Framework:
   http://www.music-ir.org/mirexdiy/
   http://www.dlib.org/dlib/december06/downie/12downie.html
   (useable ?)

MIR Software and Toolboxes

Incomplete but growing list (courtesy of Joern Loviscach):
* MARSYAS
* jAudio
* Chuck
* The Sonic Visualizer/Annotator
* CLAM
* Music-to-Knowledge (M2K)
* MIRtoolbox
* MA toolbox
* Psysound
* Praat
* IPEM
* EchoNest
* libxtract
* MuBu
* Soundspotter
* timbreID
* openSMILE
* MPEG-7 XM
* MPEG-7 Audio Encoder
* MPEG-7 Audio Analyzer
* Sphinx 4 - Java-based open-source speech recognizer  http://cmusphinx.sourceforge.net/sphinx4/#capabilities

MIR Topic Areas

From Simon Dixon, Music-IR list, Dec 2008. 

MIR Systems
- Content-based Querying
- Classification (genre/style/mood)
- Recommendation / playlist generation
- Fingerprinting / DRM
- Score following / Audio alignment
- Transcription / Annotation
- Tempo induction / Beat tracking
- Summarisation
- Streaming
- Text/web mining
- Optical music recognition
- Database systems / indexing / query languages

Human issues
- user interfaces, user models
- emotion, aesthetics
- perception, cognition
- social issues
- legal and ethical issues
- business issues
- methodological and philosophical issues

Data and metadata
- audio
- MIDI
- score
- text/web
- KR schemes, standards and protocols
- libraries and collections
- test sets and evaluation

Musical knowledge
- Melody and motives
- Harmony, chords and tonality
- Rhythm, beat, tempo and form
- Timbre, instrumentation and voice
- Genre, style and mood
- Performance
- Composition
- Ethnomusicology
Retrieved from "https://chuck.stanford.edu/mediawiki/index.php?title=MIR_workshop_2008_notes&oldid=11152"