PaddlePaddle/PaddleOCR · critical · ValueError

Missing LMDB dataset value

Error message

Missing LMDB dataset value

What it means

All LMDB reads in PaddleOCR go through _restricted_pickle_loads, which unpickles sample values with a sandboxed unpickler. If the value passed in is None (lmdb txn.get returned None because the key does not exist), it raises ValueError('Missing LMDB dataset value') before unpickling.

Solutions

  1. Verify the LMDB is complete: python -c "import lmdb; e=lmdb.open('data_dir', readonly=True, lock=False); t=e.begin(); print(t.get(b'__len__')); print(t.get(b'image-000000001') is not None)"
  2. Fix data_dir in the yml to the directory that directly contains data.mdb
  3. If keys are missing, regenerate the LMDB with tools/ (e.g. the rec det conversion scripts like gen_lmdb_dataset.py)
  4. Check read permissions and avoid concurrent writers on the same LMDB path

Example fix

# before (config)
data_dir: ./train_data/
# after (point at the actual lmdb dir containing data.mdb)
data_dir: ./train_data/rec_train_lmdb
Defensive patterns

Strategy: validation

Validate before calling

import lmdb, io, pickle
env = lmdb.open(data_dir, readonly=True, lock=False)
txn = env.begin()
assert txn.get(b'__len__') is not None, 'LMDB missing __len__; wrong dir or corrupt DB'
for i in range(min(3, pickle.loads(txn.get(b'__len__')))):
    assert txn.get(f'image-{i:09d}'.encode()) is not None, f'missing sample {i}'

Try / catch

try:
    sample = dataset[idx]
except ValueError as e:
    if 'Missing LMDB dataset value' in str(e):
        raise RuntimeError(f'LMDB at {data_dir} missing keys; regenerate it') from e
    raise

Prevention

When it happens

Trigger: Reading key b'__len__' or a sample key like b'image-000000001' from an LMDB environment where that key is absent: wrong data_dir pointing at a different LMDB, a truncated/half-written LMDB, or an index beyond num_samples.

Common situations: Pointing data_dir at the parent directory instead of the .lmdb data.mdb folder (or vice versa); interrupted LMDB conversion leaving missing keys; mixing rec-format and det-format LMDBs; file permissions causing silent read failures on network storage.

Related errors

AI-assisted analysis of PaddlePaddle/PaddleOCR@2661c7c0ef (2026-08-14). Data as JSON: /api/errors/8e17039a3d5bc9b8. Report an issue: GitHub.

Appendix: source

Thrown at ppocr/data/lmdb_dataset.py:52

    ("builtins", "frozenset"),
    ("builtins", "int"),
    ("builtins", "float"),
    ("builtins", "bool"),
}


class _RestrictedDatasetUnpickler(pickle.Unpickler):
    def find_class(self, module, name):
        if (module, name) in _ALLOWED_PICKLE_GLOBALS:
            return super().find_class(module, name)
        raise pickle.UnpicklingError(
            "Unsupported pickle payload in LMDBDataSetTableMaster dataset"
        )


def _restricted_pickle_loads(data):
    if data is None:
        raise ValueError("Missing LMDB dataset value")
    return _RestrictedDatasetUnpickler(io.BytesIO(data)).load()


class LMDBDataSet(Dataset):
    def __init__(self, config, mode, logger, seed=None):
        super(LMDBDataSet, self).__init__()

        global_config = config["Global"]
        dataset_config = config[mode]["dataset"]
        loader_config = config[mode]["loader"]
        batch_size = loader_config["batch_size_per_card"]
        data_dir = dataset_config["data_dir"]
        self.do_shuffle = loader_config["shuffle"]

        self.lmdb_sets = self.load_hierarchical_lmdb_dataset(data_dir)
        logger.info("Initialize indexes of datasets:%s" % data_dir)
        self.data_idx_order_list = self.dataset_traversal()
        if self.do_shuffle:

View on GitHub (pinned to 2661c7c0ef)