hankcs/HanLP · error · ValueError

Unsupported type of {repr(e)}

Error message

Unsupported type of {repr(e)}

What it means

dtype_of maps Python scalars to torch dtypes: bool->torch.bool, int->torch.long, float->torch.float. Any other type (str, None, list, numpy object, custom class) raises this ValueError because HanLP cannot infer a tensor dtype for it.

Source

Thrown at hanlp/utils/torch_util.py:147

def truncated_normal_(tensor, mean=0, std=1):
    size = tensor.shape
    tmp = tensor.new_empty(size + (4,)).normal_()
    valid = (tmp < 2) & (tmp > -2)
    ind = valid.max(-1, keepdim=True)[1]
    tensor.data.copy_(tmp.gather(-1, ind).squeeze(-1))
    tensor.data.mul_(std).add_(mean)
    return tensor


def dtype_of(e: Union[int, bool, float]):
    if isinstance(e, bool):
        return torch.bool
    if isinstance(e, int):
        return torch.long
    if isinstance(e, float):
        return torch.float
    raise ValueError(f'Unsupported type of {repr(e)}')


def mean_model(model: torch.nn.Module):
    return float(torch.mean(torch.stack([torch.sum(p) for p in model.parameters() if p.requires_grad])))


def main():
    start = time.time()
    print(gpus_available())
    print(time.time() - start)
    # print(gpus_available())
    # print(cuda_devices())
    # print(cuda_devices(0.1))


if __name__ == '__main__':
    main()

View on GitHub (pinned to ddb1299bdd)

Solutions

1. Inspect repr(e) in the message to find the offending sample field, then add/fix the transform that maps it to int/float ids
2. Ensure no None values reach tensorize (fill with defaults/pad values)
3. Convert numpy scalars to Python int/float first (np.int64 is not a Python int in some paths)

Example fix

# before
batch = hanlp.utils.torch_util.pad_data([['我','爱','NLP']])  # str -> ValueError
# after
ids = vocab([['我','爱','NLP']])  # tokens -> int ids first
batch = hanlp.utils.torch_util.pad_data(ids)

def is_tensorizable(sample):
    return all(isinstance(x, (bool, int, float)) and not isinstance(x, str) for x in sample)

from typing import Any, List

def is_tensorizable(batch: List[List[Any]]) -> bool:
    return all(
        isinstance(x, (bool, int, float))
        for seq in batch for x in seq
    )

try:
    batch = pad_data(data)
except ValueError as e:
    raise TypeError(f'Pipeline error: non-numeric data reached tensorization: {e}') from e

When it happens

Trigger: Tensorizing samples (tensorize/pad_data) that contain strings, None, or nested non-numeric values where flat numeric features are expected — e.g. a sample field holding raw text instead of converted ids.

Common situations: Missing a transform/convert step that turns tokens into ids before tensorization; None values from failed lookups; mixing string labels into a feature field; numpy scalars of dtype object.

Related errors

• Only supports floating point dtypes.

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/3a1bd099e982f239. Report an issue: GitHub.