Centro di Sonologia Computazionale

🎶 Released mule-torch — an unofficial PyTorch port of MULE (SF-NFNet-F0), SiriusXM/Pandora's music-audio embedding model (McCallum et al., ISMIR 2022).

No retraining: I re-implemented the architecture in pure PyTorch and transferred the original TensorFlow weights, then checked it layer by layer against the genuine TF pipeline.

✅ End-to-end clip-embedding cosine 0.9999999 vs the original
✅ ONNX backbone parity < 1e-6
✅ 62.35M params (paper: ~62.4M)
✅ Batched, GPU-native, ONNX-exportable — none of which the original Analysis pipeline does

pip install mule-torch

from mule_torch import MuleModel
emb = MuleModel.from_pretrained()(waveform)   # (B, T)@16kHz -> (B, 1728)

🤗 Weights: matteospanio/mule
💻 Code: https://github.com/matteospanio/mule-torch
📦 PyPI: https://pypi.org/project/mule-torch/

The fun bug: parity was perfect through every conv but the block output was anti-correlated (cos = −1). Cause: the learnable skip-init gains couldn't be mapped by layer name (Keras scrambles the order) — they had to be recovered from the graph.

⚠️ Unofficial, community port — not affiliated with or endorsed by the original authors. All credit to them; please cite the paper. Weights inherit CC-BY-NC-4.0.

matteospanio

authored a paper 3 months ago

BMdataset: A Musicologically Curated LilyPond Dataset

Paper • 2604.10628 • Published Apr 12 • 2

matteospanio

submitted a paper to Daily Papers 3 months ago

BMdataset: A Musicologically Curated LilyPond Dataset

Paper • 2604.10628 • Published Apr 12 • 2

matteospanio

published a model 4 months ago

csc-unipd/lilybert

Fill-Mask • 0.1B • Updated 21 days ago • 19 • 1

matteospanio

posted an update 5 months ago

Post

170

🎧 Help us evaluate AI-generated music across cultures

We are running a new online survey to collect data for a follow-up study on AI-generated music and sonic seasoning: the phenomenon where sound can influence the perception of taste.

This study builds on our previous work:
📄 Paper: A Multimodal Symphony: Integrating Taste and Sound through Generative AI (2503.02823)
🤗 Model release: csc-unipd/tasty-musicgen-small

In the previous study we introduced Tasty MusicGen, a model designed to generate music associated with taste descriptors, and showed that it can effectively produce music that evokes specific taste-related sensations.

With this new survey, we aim to expand the participant pool to create a more inclusive and cross-cultural evaluation, helping us understand how these musical cues are perceived across different linguistic and cultural backgrounds.

Participants will listen to short AI-generated music clips and evaluate the sensations they evoke.

⏱️ Takes about 10 minutes

👉 Participate here:
https://matteospanio.github.io/tasty-music-survey/

Your participation will help us better understand how generative music models interact with human perception across cultures. Thanks to everyone who participates or shares!