mudler/LocalAI · error · ValueError

dataset_source is required (path to a preprocessed dataset)

Error message

dataset_source is required (path to a preprocessed dataset)

What it means

ValueError from the liquid-audio backend's training path (_do_train): the FineTune gRPC request must carry dataset_source pointing to an already-preprocessed dataset, because the liquid_audio Trainer's LFM2DataLoader consumes preprocessed data rather than raw audio/text. An empty or missing dataset_source aborts the fine-tune job before any training starts.

Source

Thrown at backend/python/liquid-audio/backend.py:646

            job.completed = True
            print(f"Training failed: {exc}", file=sys.stderr)
            print(traceback.format_exc(), file=sys.stderr)
            job.progress_queue.put(backend_pb2.FineTuneProgressUpdate(
                job_id=job.job_id, status="failed", message=str(exc),
            ))
        finally:
            job.progress_queue.put(None)

    def _do_train(self, request, job):
        from liquid_audio import LFM2AudioModel  # noqa: F401  (sanity import)
        from liquid_audio.data.dataloader import LFM2DataLoader
        from liquid_audio.trainer import Trainer

        model_id = request.model or self.model_id or "LiquidAI/LFM2.5-Audio-1.5B"

        dataset_path = request.dataset_source
        if not dataset_path:
            raise ValueError("dataset_source is required (path to a preprocessed dataset)")

        extras = dict(request.extra_options) if request.extra_options else {}
        val_path = extras.get("val_dataset")

        # Map FineTuneRequest hyperparameters to liquid_audio.Trainer constructor args
        lr = request.learning_rate or 3e-5
        max_steps = request.max_steps or 1000
        warmup_steps = request.warmup_steps or min(100, max_steps // 10)
        batch_size = request.batch_size or 16
        save_interval = request.save_steps or max(1, max_steps // 4)

        output_dir = request.output_dir or os.path.join(
            os.environ.get("LIQUID_AUDIO_OUTPUT_DIR", "/tmp"),
            f"liquid-audio-{job.job_id}",
        )
        os.makedirs(output_dir, exist_ok=True)

        job.progress_queue.put(backend_pb2.FineTuneProgressUpdate(

View on GitHub (pinned to 44413a9d06)

Solutions

Run the preprocessing step first and pass its output directory as request.dataset_source
Verify the path exists and contains the preprocessed artifacts the LFM2DataLoader expects before submitting the job
If you meant to validate only, skip FineTune — this field is mandatory for any training run

Example fix

# before
request = backend_pb2.FineTuneRequest(base_model=model_id)  # dataset_source omitted

# after
request = backend_pb2.FineTuneRequest(
    base_model=model_id,
    dataset_source="/data/lfm2_audio_preprocessed/train",
)

Defensive patterns

Strategy: validation

Validate before calling

import os

def validate_finetune_request(req) -> None:
    ds = getattr(req, "dataset_source", "")
    if not ds:
        raise ValueError("dataset_source is required")
    if not os.path.isdir(ds):
        raise FileNotFoundError(f"preprocessed dataset not found: {ds}")

Try / catch

try:
    stub.FineTune(request)
except grpc.RpcError as e:
    if "dataset_source is required" in (e.details() or ""):
        # surface a clear UI error about the missing preprocessing step
        raise UserError("Run dataset preprocessing first") from e
    raise

Prevention

Make preprocessing a mandatory pipeline stage that outputs the path you then pass verbatim
Assert the preprocessed directory is non-empty before submitting the training job

When it happens

Trigger: Sending a FineTune request with dataset_source unset/empty; assuming the backend preprocesses raw datasets itself; passing the path under a different request field (e.g. only in extra_options).

Common situations: User points at a raw folder of wav/json files expecting preprocessing; the preprocessed output directory name was mistyped or the preprocessing job wrote elsewhere; dataset_source passed only via extra_options instead of the dedicated field.

Related errors

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/6907d935b3466155. Report an issue: GitHub.