{"record":{"id":"faaf3e5bc2e05b3d","repo":"huggingface/transformers","slug":"unable-to-create-tensor-for-key-with-overflowi","errorCode":null,"errorMessage":"Unable to create tensor for '{key}' with overflowing values of different lengths. Original error: {str(e)}","messagePattern":"Unable to create tensor for '(.+?)' with overflowing values of different lengths\\. Original error: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/feature_extraction_utils.py","lineNumber":202,"sourceCode":"        )\n\n        # Do the tensor conversion in batch\n        for key, value in self.items():\n            # Skip keys explicitly marked for no conversion\n            if skip_tensor_conversion and key in skip_tensor_conversion:\n                continue\n\n            # Skip values that are not array-like\n            if not _is_tensor_or_array_like(value):\n                continue\n\n            try:\n                if not is_tensor(value):\n                    tensor = as_tensor(value)\n                    self[key] = tensor\n            except Exception as e:\n                if key == \"overflowing_values\":\n                    raise ValueError(\n                        f\"Unable to create tensor for '{key}' with overflowing values of different lengths. \"\n                        f\"Original error: {str(e)}\"\n                    ) from e\n                raise ValueError(\n                    f\"Unable to convert output '{key}' (type: {type(value).__name__}) to tensor: {str(e)}\\n\"\n                    f\"You can try:\\n\"\n                    f\"  1. Use padding=True to ensure all outputs have the same shape\\n\"\n                    f\"  2. Set return_tensors=None to return Python objects instead of tensors\"\n                ) from e\n\n        return self\n\n    def to(self, *args, **kwargs) -> \"BatchFeature\":\n        \"\"\"\n        Send all values to device by calling `v.to(*args, **kwargs)` (PyTorch only). This should support casting in\n        different `dtypes` and sending the `BatchFeature` to a different `device`.\n\n        Args:","sourceCodeStart":184,"sourceCodeEnd":220,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/feature_extraction_utils.py#L184-L220","documentation":"During convert_to_tensors, the special 'overflowing_values' key (chunks produced by windowed feature extractors) must itself stack into a tensor. Overflow chunks commonly have different lengths, which makes stacking fail; that underlying exception is wrapped in this ValueError with the original message attached.","triggerScenarios":"Using a windowing feature extractor (e.g. Wav2Vec2 with return_attention_mask and long audio producing overflowing chunks) combined with return_tensors='pt' where the chunks have unequal lengths.","commonSituations":"Long-audio chunking pipelines; switching padding off but keeping tensor output; version changes where overflow chunk lengths stopped being normalized.","solutions":["Enable padding (padding=True with an appropriate max_length) so all overflow chunks have equal length before tensor conversion","Or set return_tensors=None and handle the list of chunks manually","Ensure truncation settings produce uniform chunk sizes when that is intended"],"exampleFix":"# before\nfe(audio, return_tensors=\"pt\")  # ragged overflow chunks\n\n# after\nfe(audio, return_tensors=\"pt\", padding=True, max_length=fe.chunk_length)","handlingStrategy":"fallback","validationCode":"def overflow_lengths_uniform(fe_out) -> bool:\n    ov = fe_out.get(\"overflowing_values\")\n    if ov is None:\n        return True\n    return len({getattr(c, \"shape\", ())[0] if hasattr(c, \"shape\") else len(c) for c in ov}) == 1","typeGuard":null,"tryCatchPattern":"try:\n    out = fe(audio, return_tensors=\"pt\")\nexcept ValueError as e:\n    if \"overflowing_values\" in str(e):\n        out = fe(audio, return_tensors=None)  # keep chunks as lists\n    else:\n        raise","preventionTips":["Pad when windowing long audio with tensor output","Handle overflowing chunks as lists unless shapes are uniform","Test with your longest real audio sample, not a short one"],"tags":["feature-extractor","batchfeature","tensor-conversion","audio","transformers"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}