unslothai/unsloth · error · ValueError

Audio VLM dataset needs 'audio' and 'text' columns, got: {da

Error message

Audio VLM dataset needs 'audio' and 'text' columns, got: {dataset.column_names}

What it means

ValueError raised in _format_audio_vlm_dataset (audio multimodal chat format, e.g. Gemma 3N) when column resolution cannot find both an audio column and a text column. Unlike CSM, both are hard requirements here — there is no default fallback — so either missing aborts formatting. The message echoes dataset.column_names for diagnosis.

Source

Thrown at studio/backend/core/training/trainer.py:1794

        return result_dataset

    def _format_audio_vlm_dataset(
        self,
        dataset,
        custom_format_mapping = None,
    ):
        """Format dataset as audio chat messages for multimodal models (e.g. Gemma 3N).

        Expects columns audio (Audio), text (str). Produces a messages column
        with system/user/assistant chat format.
        """
        from datasets import Audio

        resolved = self._resolve_audio_columns(dataset, custom_format_mapping)
        audio_col = resolved["audio_col"]
        text_col = resolved["text_col"]
        if not audio_col or not text_col:
            raise ValueError(
                f"Audio VLM dataset needs 'audio' and 'text' columns, got: {dataset.column_names}"
            )

        # Needed by the collator closure
        self._audio_vlm_audio_col = audio_col

        # Cast audio to 16kHz (standard for speech models)
        dataset = dataset.cast_column(audio_col, Audio(sampling_rate = 16000))

        def format_messages(samples):
            formatted = {"messages": []}
            for idx in range(len(samples[audio_col])):
                audio = samples[audio_col][idx]["array"]
                label = str(samples[text_col][idx])
                message = [
                    {
                        "role": "system",
                        "content": [

View on GitHub (pinned to 203007d190)

Solutions

Map your columns via custom_format_mapping (e.g. {'mp3': 'audio', 'utterance': 'text'}).
Rename columns so the resolver finds them, ensuring one audio and one text column exist.
Confirm the model type actually expects audio+text chat data; switch preprocessing path if not.

Example fix

// before
dataset  # columns: ['mp3', 'utterance']
// after
dataset = dataset.rename_column('mp3', 'audio').rename_column('utterance', 'text')

Defensive patterns

Strategy: validation

Validate before calling

def audio_vlm_ready(dataset) -> bool:
    return 'audio' in dataset.column_names and 'text' in dataset.column_names

assert audio_vlm_ready(dataset), "audio VLM needs 'audio' and 'text' columns"

Type guard

def is_audio_vlm_dataset(dataset) -> bool:
    feats = dataset.features
    return 'audio' in dataset.column_names and 'text' in dataset.column_names and 'Audio' in str(feats.get('audio'))

Try / catch

try:
    ds = trainer._format_audio_vlm_dataset(dataset, mapping)
except ValueError as e:
    if "needs 'audio' and 'text'" in str(e):
        dataset = dataset.rename_column(src_audio, 'audio').rename_column(src_text, 'text')
        ds = trainer._format_audio_vlm_dataset(dataset, mapping)

Prevention

Build chat-style audio datasets with literal 'audio' and 'text' column names.
Pass custom_format_mapping for any legacy schema instead of editing the trainer.
Assert the schema in dataset-import code so failures surface at upload time.

When it happens

Trigger: Fine-tuning an audio VLM with a dataset that has only audio (no text prompt/answer), only text, or columns with names neither the resolver nor custom_format_mapping recognizes.

Common situations: Audio classification-style datasets (label column instead of text) fed to a chat-style audio VLM; unmapped column names like 'mp3'/'utterance'; wrong model type chosen so an image or text dataset reaches the audio-VLM path.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/f6bfb7e65cb977af. Report an issue: GitHub.