PaddlePaddle/PaddleOCR · critical · ValueError

Invalid LMDB dataset length metadata

Error message

Invalid LMDB dataset length metadata

What it means

After unpickling the b'__len__' entry, the LMDB loader requires it to be a Python int (the sample count). If the stored value is a string, bytes, or any other type, it raises ValueError('Invalid LMDB dataset length metadata'), refusing to build the dataset cache entry.

Source

Thrown at ppocr/data/lmdb_dataset.py:258

        return outs


class LMDBDataSetTableMaster(LMDBDataSet):
    def load_hierarchical_lmdb_dataset(self, data_dir):
        lmdb_sets = {}
        dataset_idx = 0
        env = lmdb.open(
            data_dir,
            max_readers=32,
            readonly=True,
            lock=False,
            readahead=False,
            meminit=False,
        )
        txn = env.begin(write=False)
        num_samples = _restricted_pickle_loads(txn.get(b"__len__"))
        if not isinstance(num_samples, int):
            raise ValueError("Invalid LMDB dataset length metadata")
        lmdb_sets[dataset_idx] = {
            "dirpath": data_dir,
            "env": env,
            "txn": txn,
            "num_samples": num_samples,
        }
        return lmdb_sets

    def get_img_data(self, value):
        """get_img_data"""
        if not value:
            return None
        imgdata = np.frombuffer(value, dtype="uint8")
        if imgdata is None:
            return None
        imgori = cv2.imdecode(imgdata, 1)
        if imgori is None:
            return None

View on GitHub (pinned to 2661c7c0ef)

Solutions

1. Verify the type: print(type(_restricted_pickle_loads(txn.get(b'__len__')))) in a scratch script
2. Regenerate the LMDB with the current PaddleOCR conversion scripts, which pickle the integer count
3. Patch a custom generator to store the raw int via pickle.dumps(int(n))
4. Fall back to a label-file based dataset (data_dir + label_file) instead of LMDB if you cannot regenerate

Example fix

# before (custom generator)
env.put(b'__len__', str(n).encode())
# after
import pickle
env.put(b'__len__', pickle.dumps(int(n)))

import lmdb, pickle
n = pickle.loads(lmdb.open(data_dir, readonly=True, lock=False).begin().get(b'__len__'))
if not isinstance(n, int):
    raise SystemExit(f'LMDB __len__ has type {type(n).__name__}; regenerate the dataset')

try:
    dataset = LMDBDataSet(config, 'Train', logger)
except ValueError as e:
    if 'Invalid LMDB dataset length' in str(e):
        raise RuntimeError(f'{data_dir}: __len__ metadata not a pickled int; rebuild LMDB') from e
    raise

When it happens

Trigger: Opening an LMDB produced by a different tool where __len__ was stored as e.g. b'10000' (string) or a pickled non-int; or a hand-crafted LMDB missing proper metadata.

Common situations: Generating LMDB with a custom/old conversion script that wrote str(n) instead of the pickled int; datasets shared between frameworks (some store text metadata); partial corruption of the metadata record.

Related errors

• Missing LMDB dataset value
• {} does not exist!
• {} does not exist!
• {} does not exist!
• not found infer list {}

AI-assisted analysis of PaddlePaddle/PaddleOCR@2661c7c0ef (2026-08-14). Data as JSON: /api/errors/e3a3a29cf34472be. Report an issue: GitHub.