mudler/LocalAI · error

error: failed to detect speech\n

Error message

error: failed to detect speech\n

What it means

vad() failed because whisper_vad_detect_speech() returned false on the loaded VAD context and PCM input. This is an inference-time failure, distinct from load failures: the VAD context exists but probability computation did not complete. Returns 1 without touching segs_out.

Source

Thrown at backend/go/crispasr/cpp/crispasr_shim.cpp:200

  struct whisper_vad_context_params vcparams =
      whisper_vad_default_context_params();

  // XXX: Overridden to false in upstream due to performance?
  // vcparams.use_gpu = true;

  vctx = whisper_vad_init_from_file_with_params(model_path, vcparams);
  if (vctx == nullptr) {
    fprintf(stderr, "error: Failed to init model as VAD\n");
    return 1;
  }

  return 0;
}

int vad(float pcmf32[], size_t pcmf32_len, float **segs_out,
        size_t *segs_out_len) {
  if (!whisper_vad_detect_speech(vctx, pcmf32, pcmf32_len)) {
    fprintf(stderr, "error: failed to detect speech\n");
    return 1;
  }

  struct whisper_vad_params params = whisper_vad_default_params();
  struct whisper_vad_segments *segs =
      whisper_vad_segments_from_probs(vctx, params);
  size_t segn = whisper_vad_segments_n_segments(segs);

  // fprintf(stderr, "Got segments %zd\n", segn);

  flat_segs.clear();

  for (int i = 0; i < segn; i++) {
    flat_segs.push_back(whisper_vad_segments_get_segment_t0(segs, i));
    flat_segs.push_back(whisper_vad_segments_get_segment_t1(segs, i));
  }

  // fprintf(stderr, "setting out variables: %p=%p -> %p, %p=%zx -> %zx\n",

View on GitHub (pinned to 44413a9d06)

Solutions

1. Ensure load_model_vad succeeded (returned 0) before any vad() call.
2. Validate pcmf32_len > 0 and that the buffer holds finite float samples at whisper's expected scale.
3. Check ggml logs for compute-backend errors during detection.
4. If audio is very short, pad or skip VAD for buffers under the model's minimum window.

Example fix

// before
vad(pcm, 0, &segs, &n); // empty frame
// after: skip VAD on empty frames
if (n_pcm == 0) { *segs_out = NULL; *segs_out_len = 0; return 0; }
vad(pcm, n_pcm, &segs, &n);

if vctx == nil /* load failed or not called */ {
    return errors.New("VAD not loaded")
}
if len(pcm) == 0 {
    return nil, 0, nil // nothing to do
}

When it happens

Trigger: Calling vad(pcmf32, len) with pcmf32_len == 0, an extremely short buffer, NaN/garbage PCM data, or when the VAD context (vctx) is NULL because load_model_vad was never called or failed — detect_speech then fails immediately.

Common situations: Calling VAD before a successful load_model_vad (ordering bug in the caller); feeding an empty audio frame from an upstream silence-gate; feeding non-float or wrongly-scaled PCM.

Related errors

• error: Failed to init model as VAD\n
• Web Audio API not available
• error: failed to open CrispASR session for model\n
• error: transcription failed\n
• [moss-tts-cpp] ERROR: moss_local_tts failed\n

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/64d13868b84e9fa6. Report an issue: GitHub.