Synaptics/Google-EmbeddingGemma-2

EmbeddingGemma-2

Model Overview

EmbeddingGemma-2 is a multimodal embedding model from Google. It maps text, images, video and audio into one 768-dimensional embedding space (Matryoshka: 768, 512, 256 or 128 dimensions), so a text query can be compared directly with photos, video frames and sound clips.

This model has been optimized for the Synaptics Astra™ SL2610-Series processors with Torq NPU. This branch holds the recommended build: 8-bit weights (int8, block 32) with bf16 activations. Releases are tagged with the torq-examples version (for example v2.3.0), and the demo downloads the tag of its own version.

Build	Branch	Size	Notes
8-bit (recommended)	main (this branch), tags v2.3.0 and later	2.3 GB	Embeddings close to the original model (cosine 0.997 to fp32 over a media library); all modalities in one process
4-bit	q4 (and tag v2.2.0)	1.1 GB	Smallest; less accurate (cosine 0.976)
bf16	bf16	3.6 GB	Closest to the original model, most memory

Model Features

• Model Type: Multimodal embedding model
• Input: Text, image, video, audio
• Output: 768-d embedding

How it runs

The vision and audio encoders turn an image, video frame or sound into soft tokens in the text model's embedding space. The host writes them into the placeholder positions of the token sequence, and the text body (128 or 512 tokens, whichever holds the sequence) turns the whole sequence into the embedding.

File	Runs on	Purpose
text_body_s128.vmfb, text_body_s512.vmfb	NPU	Text body for 128 / 512 tokens (queries, images; documents, video, longer sounds)
vision_384x384.vmfb	NPU	Vision encoder: a 384x384 image or video frame -> 64 soft tokens
audio_140.vmfb … audio_1120.vmfb	NPU	Audio encoder for 1.4 / 2.8 / 5.6 / 11.2 s windows
token_embeddings.npy	CPU	Token-embedding table, memory-mapped
mel_filters.npy, mel_window.npy	CPU	Audio log-mel front end
tokenizer.json, config.json, embeddinggemma2_manifest.json	CPU	Tokenizer, model config, static shapes and NPU memory per network
examples/	—	14 CC0 / public-domain photos, sounds and videos with descriptions, used by the demo
onnx/	—	Source ONNX graphs that torq-tools builds these files from: a copy of onnx-community/embeddinggemma-2-ONNX

Performance (8-bit build, SL2619)

Time per call after a warm-up run, one network per process. CPU: ONNX Runtime 1.27 with the same 8-bit weights on the board's 2 CPU cores.

Network	NPU	CPU (ONNX Runtime)	NPU speed-up	RAM	File
text S=128	389 ms	4774 ms	12.3x	198 MB	165 MB
text S=512	2185 ms	17546 ms	8.0x	220 MB	186 MB
vision 384x384	2734 ms	18612 ms	6.8x	330 MB	303 MB
audio 1.4 s	302 ms	5793 ms	19.2x	352 MB	328 MB
audio 2.8 s	463 ms	7859 ms	17.0x	355 MB	331 MB
audio 5.6 s	935 ms	12067 ms	12.9x	361 MB	337 MB
audio 11.2 s	2041 ms	20613 ms	10.1x	381 MB	357 MB

End to end: a text query takes 0.39 s, an image 3.1 s (vision + text), a sound up to 4.9 s 0.7–1.3 s, and an 11 s sound 4.2 s.

Deployment

Usage and demos (search over a media library with text, image or sound queries, and embedding your own files) are in torq-examples under EmbeddingGemma-2:

python setup_demos.py EmbeddingGemma-2       # downloads this build and the examples
cd EmbeddingGemma-2
python src/media_search.py index             # index the example media on the NPU
python src/media_search.py search startup    # then type queries

Learn More

• Synaptics AI Developer Zone: Get started with documentation, tutorials and resources for your Edge AI journey.
• Astra Support Portal: Connect with our engineering team and community.