Lab: Can ElevenLabs match a video's temp track with use_audio_as_reference?
A planned A/B test of ElevenLabs Video to Music with and without the audio-reference flag, checking BPM, genre and SFX timing. The results are mock.
Hypothesis
The use_audio_as_reference flag is meant to pull genre, BPM, instruments and mood from a video's own audio. With it on, we expect the generated score to land within ±5 BPM of the temp track and in the same genre far more often than with a text description alone.
Setup
- Clips: six 45-second edits, each with a royalty-free temp track Vocaler owns: lo-fi hip-hop (85 BPM), synthwave (110), orchestral trailer (70), drum & bass (174), acoustic folk (100), afrobeats (105).
- Endpoint:
POST /v1/music/video-to-music. - Arm A:
description: "score that matches the energy of the edit", withuse_audio_as_reference: false. - Arm B: the same description, with
use_audio_as_reference: true. - Both arms:
add_sfx: true,separate_tracks: true, and 3 generations per clip per arm (36 in total). - Measures: BPM difference from the temp track, a blind genre-match vote from 3 listeners, SFX hits within ±100 ms of a visual event, and the number of SFX that had to be deleted.
Mock findings (illustrative only, not real results)
| Metric | Arm A (text only) | Arm B (audio reference) |
|---|---|---|
| BPM within ±5 | 6/18 | 15/18 |
| Genre match (majority vote) | 9/18 | 16/18 |
| SFX hits within ±100 ms | 71% | 73% |
| SFX deleted per clip (mean) | 2.4 | 2.2 |
Mock interpretation: the reference flag mainly improves musical match. SFX timing doesn't change, which is expected, since effects are placed from the visuals.
Follow-ups
- Test drum & bass on its own, because very high tempos may get halved.
- Compare
separate_tracksstems in a DAW mix against the combined output.
Sources
- ElevenLabs Changelog (Oct 19, 2026)elevenlabs.io/docs/changelog/2026/10/19
- ElevenLabs Blog: Introducing Instant Soundtrackselevenlabs.io/blog/introducing-instant-soundtracks