Known algorithm shortcomings
From MMS Wiki
- Rhythm is only assessed properly if the notes change. Mistakes in rhythm are very often not reported if the note played does not change.
- If creating rhythm exercises, we recommend having the student alternate between two notes instead of playing it on a single note
- When playing the exercise with a metronome coming from speakers, rhythm feedback may get confused by the metronome, resulting it missing rhythm errors.
- We recommend using headphones in metronome mode so the clicks cannot be heard in the recording.
- Algorithm does not work properly for exercises <5sec in length.
- We do not report tuning differences as long as the instrument is relatively in tune and tuning error is no more than ~40 cents
- Meaning, if the reference is tuned to 440hz, the student instrument can be tuned to anything from 430hz to 450hz
- This is actually a feature to allow hard-to-tune instruments some lee-way, but may be a problem if you need the student to be absolutely in tune.
- We do not catch off-by-octave errors (not uncommon on some wind instruments)
- The main reason for this is that the octave of the note is difficult to detect reliably.
- Algorithm gives feedback relative to absolute intonation (tuning fork), so if student drifts lower and lower (or higher and higher), she will receive a very low score
- Best way around this is to have exercises with backing and have students record with backing so they can use that as an absolute reference.
- This is less of an issue with short exercises, so another work-around is to assign songs by short sections.
- Consonants like "S", "F" and "T" produce non-pitched areas that look like noise/silence to our algorithm and can cause small yellow spots at the beginning of those syllables on the feedback line
- This is only a problem with synthesized MIDI reference files, as they do not take account of lyrics
- We recommend using real audio reference files, or asking students to sing all notes on just "A" or "O" instead of "do-re-mi" syllables
- There is NO SUPPORT WHAT SO EVER FOR DRUMS, i.e. non-pitched percussion instruments
- Glockenspiel is also known to be very problematic, even if real audio is used as reference
- Xylophone and Marimba should work, assuming the student also plays the same instrument