hiyouga/LlamaFactory · error · ValueError

Dataset converter {name} already exists.

Error message

Dataset converter {name} already exists.

What it means

register_dataset_converter raises ValueError when the given name already exists in the DATASET_CONVERTERS registry (built-ins: alpaca, sharegpt, openai). This is a plugin-API guard against silently replacing an existing converter implementation.

Source

Thrown at src/llamafactory/data/converter.py:380

            "_tools": tools,
            "_images": self._find_medias(example[self.dataset_attr.images]) if self.dataset_attr.images else None,
            "_videos": self._find_medias(example[self.dataset_attr.videos]) if self.dataset_attr.videos else None,
            "_audios": self._find_medias(example[self.dataset_attr.audios]) if self.dataset_attr.audios else None,
        }
        return output


DATASET_CONVERTERS = {
    "alpaca": AlpacaDatasetConverter,
    "sharegpt": SharegptDatasetConverter,
    "openai": OpenAIDatasetConverter,
}


def register_dataset_converter(name: str, dataset_converter: type["DatasetConverter"]) -> None:
    r"""Register a new dataset converter."""
    if name in DATASET_CONVERTERS:
        raise ValueError(f"Dataset converter {name} already exists.")

    DATASET_CONVERTERS[name] = dataset_converter


def get_dataset_converter(name: str, dataset_attr: "DatasetAttr", data_args: "DataArguments") -> "DatasetConverter":
    r"""Get a dataset converter."""
    if name not in DATASET_CONVERTERS:
        raise ValueError(f"Dataset converter {name} not found.")

    return DATASET_CONVERTERS[name](dataset_attr, data_args)


def align_dataset(
    dataset: Union["Dataset", "IterableDataset"],
    dataset_attr: "DatasetAttr",
    data_args: "DataArguments",
    training_args: "Seq2SeqTrainingArguments",
) -> Union["Dataset", "IterableDataset"]:

View on GitHub (pinned to f28afaf635)

Solutions

1. Choose a unique, namespaced converter name (e.g. 'myorg_sharegpt').
2. Make registration idempotent in your plugin: check `name in DATASET_CONVERTERS` (or wrap in try/except ValueError) before registering.
3. To override a built-in intentionally, mutate DATASET_CONVERTERS[name] directly — but prefer a new name to avoid surprising behavior.

Example fix

# before
register_dataset_converter("sharegpt", MyConverter)

# after
from llamafactory.data.converter import DATASET_CONVERTERS
if "my_sharegpt" not in DATASET_CONVERTERS:
    register_dataset_converter("my_sharegpt", MyConverter)

from llamafactory.data.converter import DATASET_CONVERTERS
if name in DATASET_CONVERTERS:
    return  # idempotent plugin registration
register_dataset_converter(name, MyConverter)

When it happens

Trigger: Calling register_dataset_converter('alpaca', MyConverter) or registering a plugin whose name collides with a built-in or with another already-registered plugin; double-importing a registration module (e.g. via reload) that registers the same name twice.

Common situations: Third-party dataset plugins with generic names; notebook workflows where a cell registering the converter is re-run; two plugins both claiming 'sharegpt'.

Related errors

• Dataset converter {name} not found.
• The model does not have a submodule named '{submodule_name}'
• Unsupported model type: {getattr(config, 'model_type')}.
• Stage does not supported: {stage}.
• Not allowed

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/5a559833f0faad00. Report an issue: GitHub.