Lastly, a huge thank you to my collaborators Vincent Lostanlen (@lostanlen.bsky.social), Emmanouil Benetos (@emmanouilb.bsky.social), and Mathieu Lagrange (mathieulagrange.github.io), without whom this project would not have been possible! 9/9
SCRAPL for the JTFS in PyTorch is available now as a Python package (`pip install scrapl-loss`) and we provide open source code, experiments, and listening samples: github.com/christhetree/scrapl/ 8/9
Lastly, a huge thank you to my collaborators Vincent Lostanlen (@lostanlen.bsky.social), Emmanouil Benetos (@emmanouilb.bsky.social), and Mathieu Lagrange (mathieulagrange.github.io), without whom this project would not have been possible! 9/9
We evaluate SCRAPL on unsupervised sound matching of granular synthesis and the Roland TR-808 drum machine and find that it accomplishes a favorable tradeoff between goodness of fit and computational efficiency (~20x faster than the JTFS with similar test accuracy). 7/9
SCRAPL with the JTFS provides an information-dense, perceptually-correlated, interpretable, and shift-invariant audio distance metric even when multiscale spectral (MSS) or embedding-based losses (e.g., CLAP, PANNs) fail on unsupervised sound matching tasks. 6/9
We apply SCRAPL to the Joint Time–Frequency Scattering (JTFS) transform, enabling it to be used as a differentiable loss function and expanding the class of synthesizers that can be effectively decoded via Differentiable Digital Signal Processing (DDSP). 5/9
We also propose a parallelizable initialization heuristic that adapts SCRAPL’s path sampling probability distribution to specific tasks by analyzing the curvature of the loss landscape, and find that it can significantly improve convergence and test accuracy. 4/9
Against this problem, we propose SCRAPL: a stochastic optimization scheme for efficient evaluation of multivariable scattering transforms. SCRAPL introduces extensions to the Adam optimizer and SAGA gradient variance reduction algorithm to stabilize its gradients. 3/9
Wavelet scattering transforms provide informative gradients for perceptual quality assessment of deep inverse problems in computer vision, speech, and audio processing. However, their many wavelets (called “paths”) make them too computationally expensive for neural network training. 2/9
SCRAPL: Scattering Transform with Random Paths for Machine Learning
I’m excited to share our #ICLR2026 paper on SCRAPL: an algorithm that makes wavelet scattering transforms usable as differentiable loss functions!
paper: openreview.net/forum?id=RuYwbd5xYa
web: christhetr.ee/scrapl/
Check out the paper, plugins, and code for more details, and join the Discord server to stay up to date. Finally, a huge thank you to my collaborators at Neutone for the amazing work they’re doing!
At a time when many AI companies are competing with artists and training on their work without permission, the SDK democratizes this technology and provides a foundation for AI tools that enhance rather than replace human creativity.
neutone.ai/blog/neutone-on-ai-and-copyright
(8/8)
To date, the SDK has powered a wide variety of applications such as neural audio effects, timbre transfer, sample generation, and stem separation, as well as seen adoption by researchers, educators, industry, and artists alike.
neutone.ai/artists
(7/8)
Since early 2022, Neutone FX has made the latest realtime neural audio models accessible to artists around the world. It includes a model browser that allows one to search for and download user models that have been shared and uploaded to the Neutone servers via the SDK. (6/8)
We provide a technical overview of the interfaces needed to accomplish this, as well as the corresponding SDK implementations. Personally, I love prototyping neural audio models in Python with the SDK, and listening to the results in the DAW seconds later after exporting. (5/8)
By encapsulating common challenges like variable buffer sizes, sample rate conversion, and delay compensation within a model-agnostic interface, our framework enables seamless interoperability between neural models and host plugins while allowing users to work entirely
in Python. (4/8)
The Neutone SDK is an open source framework that streamlines the deployment of PyTorch neural audio models for both real-time and offline applications. It enables researchers to wrap their own PyTorch models and run them in the DAW using our free host plugins FX and Gen. (3/8)
We’re also releasing the beta version of Neutone Gen, the counterpart to Neutone FX that continues to bridge the gap between audio researchers and artists. Now, you can export heavy-weight, non-realtime models using the SDK and run them in the DAW via the free Gen plugin. (2/8)
Neutone SDK: An Open Source Framework for Neural Audio Processing
We’ve finally published a paper for the Neutone SDK which I presented at AES AIMLA 2025 a few weeks ago!
arXiv: arxiv.org/abs/2508.09126
code: github.com/Neutone/neutone_sdk
discord: discord.gg/VHSMzb8Wqp
@neutone.bsky.social
Lastly, a huge thank you to my collaborator Hao Hao (github.com/gudgud96) and supervisor Josh (www.eecs.qmul.ac.uk/~josh/) for their help and contributions!
Our code is open source (github.com/christhetree/mod_discovery) and the trained synths are available as VST plugins via the @neutone.bsky.social platform and SDK.
Listening samples, visualizations, plugins, and more can be found at christhetr.ee/mod_discovery (7/7)
We evaluate our modulation discovery framework on unseen real-world modulation curves, highly modulated synthetic and real-world audio, and on white-box, gray-box, and black-box synth architectures. (6/7)
We investigate three modulation signal parameterizations:
• Framewise (Frame)
• Low-pass filtered (LPF)
• Piecewise 2D Bézier curves (Spline)
We find that LPF and Spline yield human-readable curves that trade sound-matching accuracy for interpretability. (5/7)
We apply our approach to a differentiable synthesizer inspired by the popular soft synths Serum and Vital with wavetable, filter, and envelope modulations. We also demonstrate its ability to generalize to other DDSP synth architectures. (4/7)
We propose a self-supervised neural sound-matching approach that leverages modulation extraction, constrained control signal parameterizations, and differentiable digital signal processing (DDSP) to discover the modulations present in a sound. (3/7)
Modulations are a critical part of sound design, enabling the creation of complex, evolving audio. However, finding the modulations in a sound is difficult and typical sound-matching / parameter estimation systems don’t consider the structure or routing of underlying modulations. (2/7)
Modulation Discovery with Differentiable Digital Signal Processing
This week I’ll be at @waspaa.com presenting our work on discovering synthesizer modulation signals in arbitrary audio.
arXiv: arxiv.org/abs/2510.06204
web: christhetr.ee/mod_discovery
code: github.com/christhetree/mod_discovery