Piano Note Transcription
Automatic music transcription converts performance audio signals into symbolic representations of musical scores for computational musicology. Traditionally, it is performed as separate stages: onset detection and pitch estimation, note tracking, meter detection or key estimation. Recently, some approaches integrate both acoustic and symbolic modeling into one model. These note-based methods are popular and achieve high transcription accuracy. However, these models suffer from the fact that they estimate notes without taking into account the temporal dependencies between them. To address this issue, several researchers have proposed incorporating the information about the relationship between musical notes into the acoustic models. This type of approach is known as music language models.
Musical sequences exhibit structural regularity and are often composed of segments that belong to the same note. The idea of using a recurrent neural network with a learning algorithm to model this relationship is quite appealing, since it may help to achieve better performance in the different tasks involved in music transcription, including onset detection and note tracking. However, these models still have difficulties in recognizing the correct offset of the musical notes, as they are not designed to handle such a task.
To tackle the problem of detecting the correct offset of musical notes, this paper proposes a new model for automatic piano transcription. The model is based on a recurrent neural network that uses long short term memory (LSTM), a variant of Recurrent Neural Networks that overcomes the gradient problem by selectively remembering only the relevant information from its previous inputs. The LSTM is conditioned to output one of the 88 piano keys. To map the LSTM outputs to the corresponding pitches, the authors use a one hot encoding and Mel scaled spectrograms. Compared to the state-of-the-art, this model improves by over 25.1 % on onset detection and improves by more than 85% on note-with-offset score on a newly generated test set.
Automatic Polyphonic Piano Note Transcription With Recurrent Neural Networks
Another method that tries to solve the same problem is by grouping spectral information into frequency and time domains. The authors propose a model that groups spectral features into frequency domains based on the harmonic structure of a musical piece and a rule-based framework to deal with polyphony, as well as multipitch-estimation and onset detection. This model also achieves a performance comparable to the state-of-the-art in onset detection and note-with-offset estimation on a newly generated test set.
The final model is an auto-regressive MLM that predicts the next frame activation based on both the acoustic model and the note-based MLM. This model significantly improves onset detection and note-with-offset estimates, outperforming the GBS on these metrics with the exception of recall. The improvement in recall is probably due to the fact that predictions are made at all onsets, and not only at non-blank ones, which helps to avoid the accumulation of mistakes over time.
The paper also introduces a beam search algorithm that repairs the acoustic model outputs at blank onsets. This algorithm is implemented in the inference engine of the proposed CR-GCN model and improves the overall performance of the system.