Scoring separation without ground truth
SI-SDR needs a clean reference recording. In production there is never one. Here is the model we use instead, and how far to trust it.
Every separation job we run finishes with a number attached. It looks like an SI-SDR: decibels, higher is better, the same units a separation paper would report. It is not one. A real SI-SDR needs the clean recording of each speaker to compare against, and if we had those, there would have been nothing to separate.
So the number is an estimate, produced by a model that has never seen a reference. This post is about what that model does, why it reports one figure for the pair rather than one per track, and where its answer stops being reliable.
The measurement you cannot take
Scale-invariant signal-to-distortion ratio is defined as a ratio between two things: the part of an estimated track that is the target signal, and everything else (the other speaker bleeding through, plus whatever the model invented). Computing it means projecting the estimate onto a known clean target. Offline, on a benchmark, you have that target. You built the mixture yourself.
Online, on a customer's conference recording, you have exactly one file, and it is the mixture. There is no clean per-channel truth anywhere in the system, and there never will be. That leaves two options: report nothing about quality, or estimate it.
Reporting nothing is worse than it sounds. Separation quality varies enormously with the source material (mic placement, overlap ratio, how far apart the two voices sit in timbre), and a caller who gets a poor result has no way to tell whether the model failed or the recording was never going to work. A number, even an estimated one, is the difference between "try a closer mic" and "the API is broken."
One forward pass over both tracks
The estimator is a 6.11 M-parameter model called ScoreEstimator. It does not look at either track alone. Its input is a two-channel tensor of shape (B, 2, L), holding both separated outputs together, and its output is a single scalar per item in the batch.
That shape is the central design decision, and it follows from what the question actually is. "How good is track one" is not answerable without a reference. "How cleanly are these two tracks separated from each other" is a question about the relationship between them, and the evidence for it is only visible when both are in view at once: a phrase from speaker A audible under speaker B, the same syllable appearing in both lanes at the same instant.
(B, 2, L) both separated tracks, 16 kHz │ ├─ LUFS-25 normalize on the mixture (ch1 + ch2) ├─ chunkify: 2 s window, 1 s hop ├─ log-Mel: 80 mels, n_fft 1024, hop 256, [0, 8000] Hz ├─ Conv2d(2 → 128, 1×1) + GroupNorm + GELU ← joint over both channels ├─ 3 × triple-path TF-Locoformer blocks ← time / frequency / chunk ├─ learnable-query cross-attention pools: T → F → N └─ Linear(128 → 1) │ one scalar, dB
The three pooling stages are worth a sentence. A separation failure is not spread evenly across a recording; it is concentrated in the moments where the two speakers overlap, and in the frequency bands they share. Learnable-query cross-attention pooling over time, then frequency, then chunk, lets the model weight those moments rather than averaging them away. A mean over the whole file would do exactly that, which is why a mean-pooled estimator tends to grade a mostly-clean file with three bad seconds as clean.
It is scored once for the pair and reported on both tracks, so the two read the same.
The one-line explanation shipped in the console
This surprises people, so it is worth stating plainly: if you see the same si_sdr value on spk1 and spk2 in an API response, nothing has gone wrong. There is one measurement. It is broadcast onto both stems because both stems are what it describes.
What it was trained to predict
The target is the real thing: the utterance-level permutation-invariant SI-SDR, computed offline against clean references, unclipped. Training is plain regression onto that value with a weighted mean squared error.
The weighting is the interesting part. Separator outputs in the wild are not uniformly distributed across quality. They pile up at the ends, either near-perfect or badly broken, and the interior of the range is sparse. An unweighted loss learns those two spikes and treats everything between 24 and 30 dB as rounding error. So each training sample is reweighted by inverse bin frequency across 16 bins of 2 dB over the 0–30 dB range, boosting the thin interior up to roughly 2× and stopping the loss from collapsing onto the end bins.
How well it tracks the real number
We evaluate against the offline uPIT SI-SDR on a stratified pool of separator outputs: 30 bins of 2 dB across −10 to +50 dB, capped at 1000 samples per bin, 20,855 pairs in total. The comparison point is the SI-SDR head of TorchAudio-SQUIM, the closest publicly available no-reference estimator.
ours TorchAudio-SQUIM MAE (dB) 1.84 8.03 RMSE (dB) 2.80 10.19 PCC 0.976 0.692 SRCC 0.976 0.712 bias (dB) +0.28 -3.05 ground truth avg 15.46 min -12.68 max 49.46 ours avg 15.74 min -6.78 max 46.25 SQUIM avg 12.41 min -12.67 max 30.67
Two honest qualifications. First, this is our own stratified pool of our own separator's outputs. It is not WSJ0-2mix or Libri2Mix, and we are not claiming a public-benchmark result. Second, look at the prediction range: ground truth spans −12.7 to +49.5 dB while the estimator only reaches −6.8 to +46.3. It under-covers both tails. "No hard ceiling" is directionally true next to a model that saturates at 30 dB, but it is not literally unbounded in practice.
How to read the score you get
The thresholds we ship are simple. At or above 10 dB the pair is normally usable as-is. Between 5 and 10 dB some bleed remains, and the pair is usable with care or after light editing. Below 5 dB the bleed or the artifacts are usually audible. Read them in one direction only: the estimate is biased upward on jobs that separated badly, so failing a threshold is strong evidence while clearing one is not a guarantee. The measurements for that are further down.
Those are floors, not targets. A clean two-speaker recording commonly scores in the 40 dB range, so a result well above the good threshold is normal rather than suspicious. And the score reflects the source as much as the separation: a speaker recorded quietly, or far from the mic, scores lower however well the model performed.
Compare runs; don't grade in absolutes
The estimator is most reliable when comparing runs that had similar processing, and least reliable across different ones. Three specific biases we know about: enhanced or denoised audio tends to score higher than its true separation quality, because the artifacts of enhancement look like cleanliness to the model. And a narrow-band source (8 kHz telephone audio, say) can score higher than it sounds, for the same reason.
The third is the one that matters most, and it is the reason the thresholds above are worth less than they look. The error is not symmetric across quality, which is why the +0.28 dB aggregate bias in the table above understates the risk: it averages two opposite errors. Measured against ground truth on the live API on 22 August 2026, the estimate ran roughly 1.1 dB below the true figure on jobs that separated well, and roughly 4.8 dB above it on jobs that separated badly. In the worst single case it reported 8.9 dB for a pair whose true SI-SDR was −2.0 dB, which is not merely above the warning line but inside the band we describe as usable.
The direction of that bias is the problem. It flatters exactly the jobs a caller most needs to be warned about, so a low score is informative while a middling one is not a guarantee. Treat the thresholds as a way to recognize a result that went well, not as a way to catch one that went badly, and listen to anything you intend to rely on. We would rather publish that than leave the number looking more dependable than it is.
So the number does its best work differentially. "This mic setup scores 6 dB better than that one" is a claim the estimator supports well. "This recording is a 14.2" is a claim it supports much less well, and "this recording is a 14.2 and that telephone call is a 15.1, so the call separated better" is a claim it does not support at all.
One source of run-to-run wobble
Scoring the same input twice can give slightly different numbers, and the reason is upstream of the estimator. Separation output is not bit-identical between runs, because GPU arithmetic is not deterministic, so the second run scores a very slightly different pair of tracks. The scoring step itself is deterministic: the same two tracks always yield the same score.
The label we put next to the number in the console is deliberately hedged (an estimate from a no-reference model, best for comparing runs, not an absolute grade), and that qualifier travels with every verdict we render. It is not boilerplate. It is the most accurate thing we can say about a measurement that, by construction, has nothing to measure against.
For the pipeline this scoring stage sits at the end of, see Separating the Inseparable. For the response shape it lands in, see the docs.