{"record":{"id":"8e17039a3d5bc9b8","repo":"PaddlePaddle/PaddleOCR","slug":"missing-lmdb-dataset-value","errorCode":null,"errorMessage":"Missing LMDB dataset value","messagePattern":"Missing LMDB dataset value","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"ppocr/data/lmdb_dataset.py","lineNumber":52,"sourceCode":"    (\"builtins\", \"frozenset\"),\n    (\"builtins\", \"int\"),\n    (\"builtins\", \"float\"),\n    (\"builtins\", \"bool\"),\n}\n\n\nclass _RestrictedDatasetUnpickler(pickle.Unpickler):\n    def find_class(self, module, name):\n        if (module, name) in _ALLOWED_PICKLE_GLOBALS:\n            return super().find_class(module, name)\n        raise pickle.UnpicklingError(\n            \"Unsupported pickle payload in LMDBDataSetTableMaster dataset\"\n        )\n\n\ndef _restricted_pickle_loads(data):\n    if data is None:\n        raise ValueError(\"Missing LMDB dataset value\")\n    return _RestrictedDatasetUnpickler(io.BytesIO(data)).load()\n\n\nclass LMDBDataSet(Dataset):\n    def __init__(self, config, mode, logger, seed=None):\n        super(LMDBDataSet, self).__init__()\n\n        global_config = config[\"Global\"]\n        dataset_config = config[mode][\"dataset\"]\n        loader_config = config[mode][\"loader\"]\n        batch_size = loader_config[\"batch_size_per_card\"]\n        data_dir = dataset_config[\"data_dir\"]\n        self.do_shuffle = loader_config[\"shuffle\"]\n\n        self.lmdb_sets = self.load_hierarchical_lmdb_dataset(data_dir)\n        logger.info(\"Initialize indexes of datasets:%s\" % data_dir)\n        self.data_idx_order_list = self.dataset_traversal()\n        if self.do_shuffle:","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/PaddlePaddle/PaddleOCR/blob/2661c7c0ef5c613e8f93c6e93b2e052399f0f854/ppocr/data/lmdb_dataset.py#L34-L70","documentation":"All LMDB reads in PaddleOCR go through _restricted_pickle_loads, which unpickles sample values with a sandboxed unpickler. If the value passed in is None (lmdb txn.get returned None because the key does not exist), it raises ValueError('Missing LMDB dataset value') before unpickling.","triggerScenarios":"Reading key b'__len__' or a sample key like b'image-000000001' from an LMDB environment where that key is absent: wrong data_dir pointing at a different LMDB, a truncated/half-written LMDB, or an index beyond num_samples.","commonSituations":"Pointing data_dir at the parent directory instead of the .lmdb data.mdb folder (or vice versa); interrupted LMDB conversion leaving missing keys; mixing rec-format and det-format LMDBs; file permissions causing silent read failures on network storage.","solutions":["Verify the LMDB is complete: python -c \"import lmdb; e=lmdb.open('data_dir', readonly=True, lock=False); t=e.begin(); print(t.get(b'__len__')); print(t.get(b'image-000000001') is not None)\"","Fix data_dir in the yml to the directory that directly contains data.mdb","If keys are missing, regenerate the LMDB with tools/ (e.g. the rec det conversion scripts like gen_lmdb_dataset.py)","Check read permissions and avoid concurrent writers on the same LMDB path"],"exampleFix":"# before (config)\ndata_dir: ./train_data/\n# after (point at the actual lmdb dir containing data.mdb)\ndata_dir: ./train_data/rec_train_lmdb","handlingStrategy":"validation","validationCode":"import lmdb, io, pickle\nenv = lmdb.open(data_dir, readonly=True, lock=False)\ntxn = env.begin()\nassert txn.get(b'__len__') is not None, 'LMDB missing __len__; wrong dir or corrupt DB'\nfor i in range(min(3, pickle.loads(txn.get(b'__len__')))):\n    assert txn.get(f'image-{i:09d}'.encode()) is not None, f'missing sample {i}'","typeGuard":null,"tryCatchPattern":"try:\n    sample = dataset[idx]\nexcept ValueError as e:\n    if 'Missing LMDB dataset value' in str(e):\n        raise RuntimeError(f'LMDB at {data_dir} missing keys; regenerate it') from e\n    raise","preventionTips":["Verify LMDB completeness (presence of __len__ and first/last sample keys) right after download/conversion","Keep data_dir pointing at the directory containing data.mdb","Regenerate LMDBs with the official conversion scripts only"],"tags":["lmdb","dataset","data-integrity","config"],"backgroundTag":null,"analyzedSha":"2661c7c0ef5c613e8f93c6e93b2e052399f0f854","analyzedAt":"2026-08-14T20:17:30.180Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}