{"record":{"id":"4bf0a3e714eb32e4","repo":"roboflow/supervision","slug":"merging-lazy-and-in-memory-detectiondatasets-is-no","errorCode":null,"errorMessage":"Merging lazy and in-memory DetectionDatasets is not supported.","messagePattern":"Merging lazy and in-memory DetectionDatasets is not supported\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/supervision/dataset/core.py","lineNumber":335,"sourceCode":"            >>> ds_merged = sv.DetectionDataset.merge([ds_1, ds_2])\n            >>> len(ds_merged)\n            2\n            >>> ds_merged.classes\n            ['cat', 'dog', 'person']\n\n            ```\n        \"\"\"\n\n        def is_in_memory(dataset: DetectionDataset) -> bool:\n            return len(dataset._images_in_memory) > 0 or len(dataset.image_paths) == 0\n\n        def is_lazy(dataset: DetectionDataset) -> bool:\n            return len(dataset._images_in_memory) == 0\n\n        all_in_memory = all([is_in_memory(dataset) for dataset in dataset_list])\n        all_lazy = all([is_lazy(dataset) for dataset in dataset_list])\n        if not all_in_memory and not all_lazy:\n            raise ValueError(\n                \"Merging lazy and in-memory DetectionDatasets is not supported.\"\n            )\n\n        images_in_memory = {}\n        for dataset in dataset_list:\n            images_in_memory.update(dataset._images_in_memory)\n\n        image_paths = list(\n            chain.from_iterable(dataset.image_paths for dataset in dataset_list)\n        )\n        image_paths_unique = list(dict.fromkeys(image_paths))\n        if len(image_paths) != len(image_paths_unique):\n            duplicates = find_duplicates(image_paths)\n            raise ValueError(\n                f\"Image paths {duplicates} are not unique across datasets.\"\n            )\n        image_paths = image_paths_unique\n","sourceCodeStart":317,"sourceCodeEnd":353,"githubUrl":"https://github.com/roboflow/supervision/blob/7f254d9784d4c37e0f03cd89ddee164c8db099c0/src/supervision/dataset/core.py#L317-L353","documentation":"Raised by DetectionDataset.merge() when the input datasets mix storage modes: some hold images in memory (dict of np.ndarray, or zero paths meaning fully in-memory) and some are lazy (paths only). Merge cannot produce a single consistent dataset from both modes, so it refuses rather than silently loading or dropping data.","triggerScenarios":"Calling DetectionDataset.merge([ds_in_memory, ds_lazy]) where ds_in_memory was built with images={'a.jpg': array} and ds_lazy with images=['b.jpg']. The internal is_in_memory/is_lazy predicates disagree across the list, so neither all_in_memory nor all_lazy holds.","commonSituations":"Merging a small hand-loaded dataset (arrays in memory) with a large disk-backed one; migrating old code that passed dict images (deprecated since 0.30.0) while new code passes path lists.","solutions":["Convert everything to path-based (lazy) datasets: pass a list of image paths to all constructors.","If memory allows, load all datasets with in-memory dict images so all are in-memory — though note dict-images are deprecated in 0.30.0.","Merge only same-mode datasets and handle the other with a separate dataset object."],"exampleFix":"// before\nmerged = DetectionDataset.merge([ds_dict_images, ds_path_list])  # mixed modes\n\n// after\n# normalize all to lazy path-based datasets\nmerged = DetectionDataset.merge([ds_a, ds_b])  # both built with images=[...paths...]","handlingStrategy":"validation","validationCode":"modes = {\"in_memory\" if len(ds._images_in_memory) > 0 or len(ds.image_paths) == 0 else \"lazy\" for ds in dataset_list}\nassert len(modes) == 1, f\"Cannot merge mixed dataset modes: {modes}\"\nmerged = DetectionDataset.merge(dataset_list)","typeGuard":"def all_lazy(datasets: list[DetectionDataset]) -> bool:\n    return all(len(ds._images_in_memory) == 0 for ds in datasets)","tryCatchPattern":null,"preventionTips":["Standardize on path-based (lazy) datasets everywhere — dict-images are deprecated since 0.30.0.","Wrap dataset creation in one factory so storage mode is consistent.","Never mix old dict-style code with new list-style code in the same merge."],"tags":["dataset","merge","lazy-loading","api-migration"],"backgroundTag":null,"analyzedSha":"7f254d9784d4c37e0f03cd89ddee164c8db099c0","analyzedAt":"2026-08-15T05:13:01.950Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}