6-02 - Learning to Read and Follow Music in Complete Score Sheet Images
Florian Henkel, Rainer Kelz, Gerhard Widmer
Keywords: MIR tasks, Alignment, synchronization, and score following, Domain knowledge, Machine learning/Artificial intelligence for music, MIR fundamentals and methodology, Multimodality
Abstract:
This paper addresses the task of score following in sheet music given as unprocessed images. While existing work either relies on OMR software to obtain a computer-readable score representation, or crucially relies on prepared sheet image excerpts, we propose the first system that directly performs score following in full-page, completely unprocessed sheet images. Based on incoming audio and a given image of the score, our system directly predicts the most likely position within the page that matches the audio, outperforming current state-of-the-art image-based score followers in terms of alignment precision. We also compare our method to an OMR-based approach and empirically show that it can be a viable alternative to such a system.