{"record":{"id":"5a333120f42ef4e0","repo":"huggingface/transformers","slug":"audio-transcription-requires-sequential-generation","errorCode":null,"errorMessage":"Audio transcription requires sequential generation (not CB)","messagePattern":"Audio transcription requires sequential generation \\(not CB\\)","errorType":"http","errorClass":"HTTPException","httpStatus":400,"severity":"error","filePath":"src/transformers/cli/serving/transcription.py","lineNumber":119,"sourceCode":"                \"Missing python-multipart dependency for file uploads. Install with `pip install python-multipart`\"\n            )\n\n        async with request.form() as form:\n            self._validate_request(set(form.keys()))\n            file_field = form[\"file\"]\n            if isinstance(file_field, str):\n                raise HTTPException(status_code=422, detail=\"Expected file upload, got string\")\n            file_bytes = await file_field.read()\n            model = form[\"model\"]\n            if not isinstance(model, str):\n                raise HTTPException(status_code=422, detail=\"Expected model name as string\")\n            stream = str(form.get(\"stream\", \"false\")).lower() == \"true\"\n\n        model_id_and_revision = self.model_manager.process_model_name(model)\n        audio_model, audio_processor = self.model_manager.load_model_and_processor(model_id_and_revision)\n        base_manager = self.generation_state.get_manager(model_id_and_revision)\n        if not isinstance(base_manager, GenerateManager):\n            raise HTTPException(status_code=400, detail=\"Audio transcription requires sequential generation (not CB)\")\n        gen_manager = base_manager\n        audio_inputs = self._prepare_audio_inputs(file_bytes, audio_processor, audio_model)\n\n        if stream:\n            return self._streaming(gen_manager, audio_model, audio_processor, audio_inputs)\n        return await self._non_streaming(gen_manager, audio_model, audio_processor, audio_inputs)\n\n    @staticmethod\n    def _prepare_audio_inputs(\n        file_bytes: bytes, audio_processor: \"ProcessorMixin\", audio_model: \"PreTrainedModel\"\n    ) -> dict:\n        \"\"\"Load audio bytes and convert to model inputs.\"\"\"\n        import librosa\n\n        sampling_rate = audio_processor.feature_extractor.sampling_rate\n        audio_array, _ = librosa.load(io.BytesIO(file_bytes), sr=sampling_rate, mono=True)\n        audio_inputs = audio_processor(audio_array, sampling_rate=sampling_rate, return_tensors=\"pt\").to(\n            audio_model.device","sourceCodeStart":101,"sourceCodeEnd":137,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cli/serving/transcription.py#L101-L137","documentation":"HTTP 400 from the transcription handler when the model's registered manager in GenerationState is not a GenerateManager, i.e. the server is running the model in continuous-batching (CB) mode. The transcription path is only implemented on the sequential generation manager, so CB-served models cannot serve /v1/audio/transcriptions.","triggerScenarios":"Starting the server with continuous batching enabled (e.g. --cb / CB flags) and then POSTing to /v1/audio/transcriptions with a model handled by the CB engine.","commonSituations":"A single serving deployment serving both chat completions (CB for throughput) and audio transcriptions; enabling CB for benchmarking then forgetting it blocks transcription.","solutions":["Restart the serving CLI without the continuous batching flag so a sequential GenerateManager handles the model","Run a second server instance without CB dedicated to audio transcription","Route transcription traffic to a deployment configured for sequential generation"],"exampleFix":"# before\n$ transformers serve --cb --model openai/whisper-large-v3\n# after\n$ transformers serve --model openai/whisper-large-v3","handlingStrategy":"validation","validationCode":"# before shipping transcription traffic, probe capability\nhealth = requests.get(f'{base}/health').json()  # or config endpoint exposing cb mode\nif health.get('cb_enabled'):\n    route_transcription_to_sequential_server()","typeGuard":null,"tryCatchPattern":"if resp.status_code == 400 and 'sequential generation' in resp.text:\n        raise RuntimeError('transcription unsupported in CB mode; use a non-CB deployment') from None","preventionTips":["Do not enable continuous batching on deployments that must serve /v1/audio/transcriptions","Keep a separate sequential server instance for audio workloads"],"tags":["serving","audio","continuous-batching","http-400"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}