MuseGAN

As explained in Learn Generative AI with PyTorch

A music-generation GAN variant in this chapter that produces multidimensional representations of musical pieces for later conversion to MIDI.

MuseGAN generates multitrack music by treating a complete piece as a multidimensional object, much like an image. A generator creates the object; a critic scores it against real music. Unlike a sequence model that predicts the next musical event, this approach generates the piece as a structured whole. section 13.0

A training piece has shape (4, 2, 16, 84): four tracks, two bars per track, 16 steps per bar, and 84 possible pitches per step. Bach chorales provide the training examples, so the intended output resembles their style. section 13.6 section 13.3

Four random latent vectors divide control over the piece:

• Chords influence the tracks together and can change between bars. section 13.2 section 13.4.2
• Style remains shared across tracks and constant across both bars. section 13.2 section 13.4.2
• Melody gives tracks separate control and can change between bars. section 13.2 section 13.4.2
• Groove gives each track its own control while remaining constant across bars. section 13.2 section 13.4.2

The generator expands chords and melody along the bar dimension with temporal networks, then uses bar generators to construct one track-bar segment at a time. The critic instead applies convolutional layers to a full (4, 2, 16, 84) piece and returns an unrestricted score, not a real-or-fake classification. section 13.4.2 section 13.4.1

During training, the critic scores real and generated pieces, while the generator learns to obtain higher scores. A gradient penalty uses an interpolation between real and generated pieces to help stabilize the critic’s training. section 13.2.2

Figure 13.6 A diagram of the steps involved in training MuseGAN to generate music. The generator produces a fake music piece by drawing four random noise vectors from the latent space (top left) and presents it to the critic (middle). The critic evaluates the piece and assigns a rating. A high rating suggests that the piece is likely from the training dataset, while a lower rating indicates that the piece is likely fake (generated by the generator). Additionally, an interpolated music piece created from a mix of real and fake samples (top left) is presented to the critic. The training process incorporates a gradient penalty based on the critic’s rating of this interpolated piece, which is added to the total loss. The ratings are then compared to the ground truth, allowing both the critic and the generator to learn from these evaluations. After numerous training iterations, the generator becomes proficient at producing music pieces that are virtually indistinguishable from real samples.

Follow the four latent inputs through the generator, then compare how real, generated, and interpolated music enter the critic. section 13.2.2

Generate a playable piece

Draw five sets of random chords and style vectors with shape (5, 32), and melody and groove vectors with shape (5, 4, 32). Pass them to a trained generator, then use convert_to_midi() to join the five resulting music objects into one MIDI file. Playback offers a direct comparison with the training music; new random vectors produce a different result. section 13.5.2

How Learn Generative AI with PyTorch covers it

How deep it goes

Works through the music shape, four latent inputs, generator and critic networks, training feedback, and MIDI export.

Examples

Uses Bach chorales for training and joins five generated music objects into a playable MIDI file.

What it assumes

Builds on GAN and convolutional image-generation ideas while defining tracks, bars, steps, and pitch for the music object.

What you can do after

Build it from scratch as a generator–critic model, train it on chorales, and export generated pieces as MIDI.

How it connects

Connects generative adversarial networks, Wasserstein critics, convolutional networks, latent vectors, symbolic music representation, and MIDI.

liveBookmanning.com

Read Learn Generative AI with PyTorch on liveBook Get the book at manning.com

Related concepts

• discriminatorpart of · A discriminator is the model component of a generative adversarial network that distinguishes real data from data generated by its generator.
• generatorpart of · A generator is a computational component or construct that produces data or values from inputs, representations, context, or specified constraints.
• JSB Chorales datasetuses · A Bach chorales dataset arranged for four tracks and used as training data for music generation models.
• generative adversarial networkkind of · A generative adversarial network is a generative model with a generator and discriminator trained in competition, enabling the generator to produce samples resembling a target dataset or distribution.
• latent spaceuses · A latent space is a learned representational space in which numerical coordinates encode information or features inferred from observed data.
• music generationimplements · Music generation is the computational creation of musical sequences, performances, or audio by algorithmic or learned models, often from prompts or other inputs.

This explainer was generated by Manning's knowledge graph from the text of Learn Generative AI with PyTorch. The section links point to the parts of the book it draws on.