NSynth (Magenta / Google Research)

NSynth (Magenta / Google Research) is an open-source neural audio synthesis project launched by Google’s Magenta team inside Google Brain. Announced in 2017 as part of Magenta’s research into machine learning for music and art, NSynth trains neural encoders and decoders on thousands of single-note recordings to learn continuous latent spaces of timbre. Its core value proposition is letting users synthesize new instrument sounds by interpolating between learned embeddings or by passing a single audio sample through the model to generate a transformed waveform. NSynth is positioned as a research and creative tool rather than a commercial product, published with model checkpoints, code, and demos on the Magenta website and GitHub for reproducible audio research and experimentation. Because it’s distributed under an open license, anyone with TensorFlow experience can run and modify the models.

NSynth’s technical feature set centers on sample-level synthesis and latent-space manipulation. The original NSynth model uses an encoder to map raw audio to a 16- or 64-dimensional embedding and a WaveNet-style autoregressive decoder to generate audio waveforms at 16 kHz (original research used 16 kHz). The tool provides utilities to interpolate between two embeddings (cross-synthesis), perform “additive” blending of timbres, and apply learned timbre transformations to single notes. Magenta also supplies preprocessed datasets (the NSynth dataset of ~300k labeled notes) and Jupyter notebooks that show how to generate sound from checkpointed models. Users can run the TensorFlow checkpoints locally or explore the interactive web demo that lets you drag between instrument embeddings to hear real-time interpolation examples. The project emphasizes reproducible pipelines: data loaders, training scripts, and inference code are included in the GitHub repo.

NSynth itself is distributed freely (open-source) so there is no paid hosted plan provided by Magenta/Google. The project provides downloadable model checkpoints and the NSynth dataset (about 305,979 musical notes) at no charge. Costs for users come from compute: running the WaveNet decoder for real-time synthesis typically requires a modern CPU for offline rendering or a GPU for faster inference; cloud GPU costs vary by provider and are not billed by Magenta. There are no tiered subscriptions, enterprise SLA, or commercial licensing fees from Magenta for the code; however, commercial projects must follow the repository license and any sample copyrights. In short: the software and models are free to download, but operational costs (compute, storage, engineering time) are borne by the user.

Practically, NSynth is used by sound designers creating novel single-note instruments, researchers studying timbre representations, and developers prototyping audio ML applications. For example: a sound designer at a game studio might use NSynth to produce 100 unique weapon sonic textures by interpolating between acoustic and synthetic instrument embeddings. An academic researcher in music cognition could leverage the NSynth dataset and checkpoints to run controlled experiments on perceptual similarity across timbres. NSynth is less of a plug-and-play DAW plugin and more of a research-grade synthesis engine; users who want end-to-end song generation or commercial cloud APIs may prefer competitors like OpenAI’s Jukebox or commercial synth plugins, but NSynth remains unique for dataset-provided latent timbre interpolation and raw waveform generation.