hiyouga/LlamaFactory · error · ValueError

Invalid index type {type(index)}.

Error message

Invalid index type {type(index)}.

What it means

The concatenated dataset wrapper supports only slice and list[int] indexing on its data_index. Any other index type (int, numpy scalar, tensor, tuple) raises this ValueError from _get_subset_of_data.

Source

Thrown at src/llamafactory/v1/plugins/data_plugins/loader.py:108

def select_data_sample(
    data_index: list[tuple[str, int]], index: slice | list[int] | Any
) -> tuple[str, int] | list[tuple[str, int]]:
    """Select dataset samples.

    Args:
        data_index (list[tuple[str, int]]): List of (dataset_name, sample_index).
        index (Union[slice, list[int], Any]): Index of dataset samples.

    Returns:
        Union[tuple[str, int], list[tuple[str, int]]]: Selected dataset samples.
    """
    if isinstance(index, slice):
        return [data_index[i] for i in range(*index.indices(len(data_index)))]
    elif isinstance(index, list):
        return [data_index[i] for i in index]
    else:
        raise ValueError(f"Invalid index type {type(index)}.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Wrap single indices in a list: dataset[[i]] instead of dataset[i].
2. Use slices for ranges: dataset[start:end].
3. Convert numpy/tensor indices to a Python list of ints before indexing.
4. For iteration, rely on the standard DataLoader which emits valid index lists.

Example fix

# before
sample = dataset[3]

# after
sample = dataset[[3]][0]

def norm_index(i):
    if isinstance(i, slice):
        return i
    if hasattr(i, 'item'):
        i = i.item()
    if isinstance(i, int):
        return [i]
    if isinstance(i, (list, tuple)):
        return [int(x) for x in i]
    raise TypeError(f'unsupported index {type(i)}')

def is_valid_index(i) -> bool:
    """True for slice or list[int] indices accepted by the v1 dataset."""
    return isinstance(i, slice) or (isinstance(i, list) and all(isinstance(x, int) for x in i))

try:
    rows = dataset[idx]
except ValueError as e:
    if 'Invalid index type' in str(e):
        rows = dataset[norm_index(idx)]
    else:
        raise

When it happens

Trigger: Calling dataset[0] with a plain int, dataset[np.int64(3)], or indexing with a torch tensor / tuple, instead of dataset[[0]] or dataset[0:1].

Common situations: Custom training loops or debugging code that uses int indexing out of habit; PyTorch DataLoader samplers that hand over numpy/tensor indices; iterating with random single indices.

Related errors

• The length of packed example should be identical to the cuto
• Input must be string, set[str] or dict[str, str], got {type(
• Unexpected role: {}
• GLM-4 does not support parallel functions.
• Streaming mode should have an integer val size.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/8c980f14cdf57f17. Report an issue: GitHub.