Pulse

Pulse/Lab

Experiment1 min read

Lab: Can ElevenLabs match a video's temp track with use_audio_as_reference?

A planned A/B test of ElevenLabs Video to Music with and without the audio-reference flag, checking BPM, genre and SFX timing. The results are mock.

Hypothesis

The use_audio_as_reference flag is meant to pull genre, BPM, instruments and mood from a video's own audio. With it on, we expect the generated score to land within ±5 BPM of the temp track and in the same genre far more often than with a text description alone.

Setup

  • Clips: six 45-second edits, each with a royalty-free temp track Vocaler owns: lo-fi hip-hop (85 BPM), synthwave (110), orchestral trailer (70), drum & bass (174), acoustic folk (100), afrobeats (105).
  • Endpoint: POST /v1/music/video-to-music.
  • Arm A: description: "score that matches the energy of the edit", with use_audio_as_reference: false.
  • Arm B: the same description, with use_audio_as_reference: true.
  • Both arms: add_sfx: true, separate_tracks: true, and 3 generations per clip per arm (36 in total).
  • Measures: BPM difference from the temp track, a blind genre-match vote from 3 listeners, SFX hits within ±100 ms of a visual event, and the number of SFX that had to be deleted.

Mock findings (illustrative only, not real results)

MetricArm A (text only)Arm B (audio reference)
BPM within ±56/1815/18
Genre match (majority vote)9/1816/18
SFX hits within ±100 ms71%73%
SFX deleted per clip (mean)2.42.2

Mock interpretation: the reference flag mainly improves musical match. SFX timing doesn't change, which is expected, since effects are placed from the visuals.

Follow-ups

  • Test drum & bass on its own, because very high tempos may get halved.
  • Compare separate_tracks stems in a DAW mix against the combined output.

Sources

  1. ElevenLabs Changelog (Oct 19, 2026)elevenlabs.io/docs/changelog/2026/10/19
  2. ElevenLabs Blog: Introducing Instant Soundtrackselevenlabs.io/blog/introducing-instant-soundtracks

Keep reading

Latest
↑ ↓ to move · Enter to open · Esc to close