PaddlePaddle/PaddleOCR · error · ValueError

Please set inference model dir in Global.inference_model or

Error message

Please set inference model dir in Global.inference_model or Global.pretrained_model for post-quantization

What it means

ValueError from deploy/slim/quantization/quant_kl.py during post-training quantization setup. When Global.inference_model is not set, the script derives a directory from Global.pretrained_model and requires inference.pdmodel + inference.pdiparams to exist there; if either file is missing it aborts asking you to set an inference model dir.

Source

Thrown at deploy/slim/quantization/quant_kl.py:147

        config["Eval"]["loader"]["num_workers"] = 0
        valid_dataloader = build_dataloader(config, "Eval", device, logger)
        if is_layoutxlm_ser:
            train_dataloader = valid_dataloader
    else:
        valid_dataloader = None

    paddle.enable_static()
    exe = paddle.static.Executor(device)

    if "inference_model" in global_config.keys():  # , 'inference_model'):
        inference_model_dir = global_config["inference_model"]
    else:
        inference_model_dir = os.path.dirname(global_config["pretrained_model"])
        if not (
            os.path.exists(os.path.join(inference_model_dir, "inference.pdmodel"))
            and os.path.exists(os.path.join(inference_model_dir, "inference.pdiparams"))
        ):
            raise ValueError(
                "Please set inference model dir in Global.inference_model or Global.pretrained_model for post-quantization"
            )

    if is_layoutxlm_ser:
        generator = sample_generator_layoutxlm_ser(train_dataloader)
    else:
        generator = sample_generator(train_dataloader)

    paddleslim.quant.quant_post_static(
        executor=exe,
        model_dir=inference_model_dir,
        model_filename="inference.pdmodel",
        params_filename="inference.pdiparams",
        quantize_model_path=global_config["save_inference_dir"],
        sample_generator=generator,
        save_model_filename="inference.pdmodel",
        save_params_filename="inference.pdiparams",
        batch_size=1,

View on GitHub (pinned to 2661c7c0ef)

Solutions

Export an inference model first (paddleocr export or tools/export_model.py) so inference.pdmodel and inference.pdiparams exist in a directory.
Set `Global.inference_model: <dir>` in the quantization config to that directory.
Or point Global.pretrained_model at a prefix whose directory contains inference.pdmodel/inference.pdiparams.
Use absolute paths in the config to avoid cwd-dependent resolution.

Example fix

// before
Global:
  pretrained_model: ./output/best_accuracy

// after
# after running the model export step:
Global:
  inference_model: ./output/inference

Defensive patterns

Strategy: validation

Validate before calling

import os

def inference_model_ready(global_config: dict) -> bool:
    d = global_config.get("inference_model") or os.path.dirname(
        global_config.get("pretrained_model", "")
    )
    return os.path.isfile(os.path.join(d, "inference.pdmodel")) and os.path.isfile(
        os.path.join(d, "inference.pdiparams")
    )

Try / catch

try:
    run_quant_kl(global_config)
except ValueError as e:
    if "inference model dir" in str(e):
        log.error("export an inference model first, then set Global.inference_model")
    raise

Prevention

Always run the model export step before KL post-quantization; checkpoints alone are not quantizable here.
Prefer Global.inference_model with an absolute path over inferring directories from pretrained_model.
Verify inference.pdmodel and inference.pdiparams both exist in the target dir as a CI precheck.

When it happens

Trigger: Slim config sets only pretrained_model pointing at a checkpoint prefix (e.g. best_accuracy) instead of a saved inference model; inference model saved under different filenames; relative paths resolved against the wrong working directory.

Common situations: Users assuming a trained checkpoint can be quantized directly — KL post-quantization needs an exported inference model produced by paddle.jit.save / export tools; path typos; running from a different cwd so relative pretrained_model paths miss.

Related errors

AI-assisted analysis of PaddlePaddle/PaddleOCR@2661c7c0ef (2026-08-14). Data as JSON: /api/errors/49f5b21ce4c35261. Report an issue: GitHub.