{"record":{"id":"3a1bd099e982f239","repo":"hankcs/HanLP","slug":"unsupported-type-of-repr-e","errorCode":null,"errorMessage":"Unsupported type of {repr(e)}","messagePattern":"Unsupported type of (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"hanlp/utils/torch_util.py","lineNumber":147,"sourceCode":"\ndef truncated_normal_(tensor, mean=0, std=1):\n    size = tensor.shape\n    tmp = tensor.new_empty(size + (4,)).normal_()\n    valid = (tmp < 2) & (tmp > -2)\n    ind = valid.max(-1, keepdim=True)[1]\n    tensor.data.copy_(tmp.gather(-1, ind).squeeze(-1))\n    tensor.data.mul_(std).add_(mean)\n    return tensor\n\n\ndef dtype_of(e: Union[int, bool, float]):\n    if isinstance(e, bool):\n        return torch.bool\n    if isinstance(e, int):\n        return torch.long\n    if isinstance(e, float):\n        return torch.float\n    raise ValueError(f'Unsupported type of {repr(e)}')\n\n\ndef mean_model(model: torch.nn.Module):\n    return float(torch.mean(torch.stack([torch.sum(p) for p in model.parameters() if p.requires_grad])))\n\n\ndef main():\n    start = time.time()\n    print(gpus_available())\n    print(time.time() - start)\n    # print(gpus_available())\n    # print(cuda_devices())\n    # print(cuda_devices(0.1))\n\n\nif __name__ == '__main__':\n    main()\n","sourceCodeStart":129,"sourceCodeEnd":165,"githubUrl":"https://github.com/hankcs/HanLP/blob/ddb1299bddff079e447af52ec12549c50636bfa8/hanlp/utils/torch_util.py#L129-L165","documentation":"dtype_of maps Python scalars to torch dtypes: bool->torch.bool, int->torch.long, float->torch.float. Any other type (str, None, list, numpy object, custom class) raises this ValueError because HanLP cannot infer a tensor dtype for it.","triggerScenarios":"Tensorizing samples (tensorize/pad_data) that contain strings, None, or nested non-numeric values where flat numeric features are expected — e.g. a sample field holding raw text instead of converted ids.","commonSituations":"Missing a transform/convert step that turns tokens into ids before tensorization; None values from failed lookups; mixing string labels into a feature field; numpy scalars of dtype object.","solutions":["Inspect repr(e) in the message to find the offending sample field, then add/fix the transform that maps it to int/float ids","Ensure no None values reach tensorize (fill with defaults/pad values)","Convert numpy scalars to Python int/float first (np.int64 is not a Python int in some paths)"],"exampleFix":"# before\nbatch = hanlp.utils.torch_util.pad_data([['我','爱','NLP']])  # str -> ValueError\n# after\nids = vocab([['我','爱','NLP']])  # tokens -> int ids first\nbatch = hanlp.utils.torch_util.pad_data(ids)","handlingStrategy":"type-guard","validationCode":"def is_tensorizable(sample):\n    return all(isinstance(x, (bool, int, float)) and not isinstance(x, str) for x in sample)","typeGuard":"from typing import Any, List\n\ndef is_tensorizable(batch: List[List[Any]]) -> bool:\n    return all(\n        isinstance(x, (bool, int, float))\n        for seq in batch for x in seq\n    )","tryCatchPattern":"try:\n    batch = pad_data(data)\nexcept ValueError as e:\n    raise TypeError(f'Pipeline error: non-numeric data reached tensorization: {e}') from e","preventionTips":["Always run token->id transforms before tensorize","Replace None with 0/pad ids during preprocessing","Cast numpy scalars: int(x)/float(x)"],"tags":["tensor","dtype","data-pipeline"],"backgroundTag":"unsupported-type-for-tensor","analyzedSha":"ddb1299bddff079e447af52ec12549c50636bfa8","analyzedAt":"2026-08-27T03:36:54.287Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}