Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) are local in space and time and closely related to a biological model of memory in the prefrontal cortex. Not only are they more biologically plausible than previous artificial RNNs, they also outperformed them on many artificially generated sequential processing tasks.

Abstract. In this paper, we describe how we can generate music by simulating moves of artificial ants on a graph where vertices represent notes and edges represent possible transitions between notes. As ants can deposit pheromones on edges, they collectively build a melody which is a sequence of Midi events. Different parameter settings are tested to produce different styles of generated music with several instruments. We also introduce a mechanism that takes into account music files to initialize the pheromone matrix. 1

This paper addresses the challenge of learning global musical structure from databases of music sequences. We introduce a music-specific sequence learner that combines an LSTM recurrent neural network with an autocorrelationbased predictor of metrical structure. The model is able to learn arbitrary long-timescale correlations in music but is biased towards finding correlations that are aligned with the meter of the piece. This biasing allows the model to work with low learning capacity and thus to avoid overfitting. In a set of simulations we show that the model can learn the global temporal structure of a musical style by simply trying to predict the next note in a set of pieces selected from that style. To test whether global structure has in fact been been learned, we use the model to generate new pieces of music in that style. In a discussion of the model we highlight its sensitivity to three distinct levels of temporal order in music corresponding to local structure, long-timescale metrical structure and long-timescale non-metrical structure. 1

Abstract. Humans are able to perform a large variety of periodic activities in different modes, for instance cyclic rehearsal of phone numbers, humming a melody sniplet over and over again. These performances are, to a certain degree, robust against perturbations, and it often suffices to present a new pattern a few times only until it can be “picked up”. From an abstract mathematical perspective, this implies that the brain, as a dynamical system, (1) hosts a very large number of cyclic attractors, such that (2) if the system is driven by external input with a cyclic motif, it can entrain to a closely corresponding attractor in a very short time. This chapter proposes a simple recurrent neural network architecture which displays these dynamical phenomena. The model builds on echo state networks (ESNs), which have recently become popular in machine learning and computational neuroscience.

Modeling long-term dependencies in time series has proved very difficult to achieve with traditional machine learning methods. This problem occurs when considering music data. In this paper, we introduce a model for rhythms based on the distributions of distances between subsequences. A specific implementation of the model when considering Hamming distances over a simple rhythm representation is described. The proposed model consistently outperforms a standard Hidden Markov Model in terms of conditional prediction accuracy on two different music databases. 1.

Chord progressions are the building blocks from which tonal music is constructed. Inferring chord progressions is thus an essential step towards modeling long term dependencies in music. In this paper, a distributed representation for chords is designed such that Euclidean distances roughly correspond to psychoacoustic dissimilarities. Estimated probabilities of chord substitutions are derived from this representation and are used to introduce smoothing in graphical models observing chord progressions. Parameters in the graphical models are learnt with the EM algorithm and the classical Junction Tree algorithm is used for inference. Various model architectures are compared in terms of conditional out-of-sample likelihood. Both perceptual and statistical evidence show that binary trees related to meter are well suited to capture chord dependencies.

Abstract — Evolutionary hypernetworks (EHNs) are recently introduced models for learning higher-order probabilistic relations of data by an evolutionary self-organizing process. We present a method that enables EHNs to learn and generate music from examples. Short-term and long-term sequential patterns can be extracted and combined to generate music with various styles by our method. Based on a music corpus consisting of several genres and artists, an EHN generates genre-specific or artist-dependent music fragments when a fraction of score is given as a cue. Our method shows about 88% of success rate in partial music completion task. By inspecting hyperedges in the trained hypernetworks, we can extract a set of arguments that constitutes melodic structures in music. I.

submitted for publication Abstract. Modeling long-term dependencies in time series has proved very difficult to achieve with traditional machine learning methods. This problem occurs when considering music data. In this paper, we introduce generative models for melodies. We decompose melodic modeling into two subtasks. We first propose a rhythm model based on the distributions of distances between subsequences. Then, we define a generative model for melodies given chords and rhythms based on modeling sequences of Narmour features. The rhythm model consistently outperforms a standard Hidden Markov Model in terms of conditional prediction accuracy on two different music databases. Using a similar evaluation procedure, the proposed melodic model consistently outperforms an Input/Output Hidden Markov Model. Furthermore, sampling these models given appropriate musical contexts generates realistic melodies. 2 IDIAP–RR 08-51 1

Abstract. Modeling long-term dependencies in time series has proved very difficult to achieve with traditional machine learning methods. This problem occurs when considering music data. In this paper, we introduce a model for rhythms based on the distributions of distances between subsequences. A specific implementation of the model when considering Hamming distances over a simple rhythm representation is described. The proposed model consistently outperforms a standard Hidden Markov Model in terms of conditional prediction accuracy on two different music databases. 2 IDIAP–RR 08-33 1

submitted for publication Abstract. Chord progressions are the building blocks from which tonal music is constructed. The choice of a particular representation for chords has a strong impact on statistical modeling of the dependence between chord symbols and the actual sequences of notes in polyphonic music. Melodic prediction is used in this paper as a benchmark task to evaluate the quality of four chord representations using two probabilistic model architectures derived from Input/Output Hidden Markov Models (IOHMMs).2 IDIAP–RR 08-50 1