{"record":{"id":"f6bfb7e65cb977af","repo":"unslothai/unsloth","slug":"audio-vlm-dataset-needs-audio-and-text-columns","errorCode":null,"errorMessage":"Audio VLM dataset needs 'audio' and 'text' columns, got: {dataset.column_names}","messagePattern":"Audio VLM dataset needs 'audio' and 'text' columns, got: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/trainer.py","lineNumber":1794,"sourceCode":"        return result_dataset\n\n    def _format_audio_vlm_dataset(\n        self,\n        dataset,\n        custom_format_mapping = None,\n    ):\n        \"\"\"Format dataset as audio chat messages for multimodal models (e.g. Gemma 3N).\n\n        Expects columns audio (Audio), text (str). Produces a messages column\n        with system/user/assistant chat format.\n        \"\"\"\n        from datasets import Audio\n\n        resolved = self._resolve_audio_columns(dataset, custom_format_mapping)\n        audio_col = resolved[\"audio_col\"]\n        text_col = resolved[\"text_col\"]\n        if not audio_col or not text_col:\n            raise ValueError(\n                f\"Audio VLM dataset needs 'audio' and 'text' columns, got: {dataset.column_names}\"\n            )\n\n        # Needed by the collator closure\n        self._audio_vlm_audio_col = audio_col\n\n        # Cast audio to 16kHz (standard for speech models)\n        dataset = dataset.cast_column(audio_col, Audio(sampling_rate = 16000))\n\n        def format_messages(samples):\n            formatted = {\"messages\": []}\n            for idx in range(len(samples[audio_col])):\n                audio = samples[audio_col][idx][\"array\"]\n                label = str(samples[text_col][idx])\n                message = [\n                    {\n                        \"role\": \"system\",\n                        \"content\": [","sourceCodeStart":1776,"sourceCodeEnd":1812,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/trainer.py#L1776-L1812","documentation":"ValueError raised in _format_audio_vlm_dataset (audio multimodal chat format, e.g. Gemma 3N) when column resolution cannot find both an audio column and a text column. Unlike CSM, both are hard requirements here — there is no default fallback — so either missing aborts formatting. The message echoes dataset.column_names for diagnosis.","triggerScenarios":"Fine-tuning an audio VLM with a dataset that has only audio (no text prompt/answer), only text, or columns with names neither the resolver nor custom_format_mapping recognizes.","commonSituations":"Audio classification-style datasets (label column instead of text) fed to a chat-style audio VLM; unmapped column names like 'mp3'/'utterance'; wrong model type chosen so an image or text dataset reaches the audio-VLM path.","solutions":["Map your columns via custom_format_mapping (e.g. {'mp3': 'audio', 'utterance': 'text'}).","Rename columns so the resolver finds them, ensuring one audio and one text column exist.","Confirm the model type actually expects audio+text chat data; switch preprocessing path if not."],"exampleFix":"// before\ndataset  # columns: ['mp3', 'utterance']\n// after\ndataset = dataset.rename_column('mp3', 'audio').rename_column('utterance', 'text')","handlingStrategy":"validation","validationCode":"def audio_vlm_ready(dataset) -> bool:\n    return 'audio' in dataset.column_names and 'text' in dataset.column_names\n\nassert audio_vlm_ready(dataset), \"audio VLM needs 'audio' and 'text' columns\"","typeGuard":"def is_audio_vlm_dataset(dataset) -> bool:\n    feats = dataset.features\n    return 'audio' in dataset.column_names and 'text' in dataset.column_names and 'Audio' in str(feats.get('audio'))","tryCatchPattern":"try:\n    ds = trainer._format_audio_vlm_dataset(dataset, mapping)\nexcept ValueError as e:\n    if \"needs 'audio' and 'text'\" in str(e):\n        dataset = dataset.rename_column(src_audio, 'audio').rename_column(src_text, 'text')\n        ds = trainer._format_audio_vlm_dataset(dataset, mapping)","preventionTips":["Build chat-style audio datasets with literal 'audio' and 'text' column names.","Pass custom_format_mapping for any legacy schema instead of editing the trainer.","Assert the schema in dataset-import code so failures surface at upload time."],"tags":["audio","vlm","multimodal","dataset","validation"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}