{"record":{"id":"5438539f3a9fba06","repo":"nautechsystems/nautilus_trader","slug":"intervals-are-not-disjoint-after-extending-a-file","errorCode":null,"errorMessage":"Intervals are not disjoint after extending a file","messagePattern":"Intervals are not disjoint after extending a file","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/persistence/src/backend/catalog.rs","lineNumber":1288,"sourceCode":"        let start = start.as_u64();\n        let end = end.as_u64();\n\n        for interval in intervals {\n            if interval.0 == end + 1 {\n                // Extend backwards: new file covers [start, interval.1]\n                self.rename_parquet_file(&directory, interval.0, interval.1, start, interval.1)?;\n                break;\n            } else if interval.1 == start - 1 {\n                // Extend forwards: new file covers [interval.0, end]\n                self.rename_parquet_file(&directory, interval.0, interval.1, interval.0, end)?;\n                break;\n            }\n        }\n\n        let intervals = self.get_directory_intervals(&directory)?;\n\n        if !are_intervals_disjoint(&intervals) {\n            anyhow::bail!(\"Intervals are not disjoint after extending a file\");\n        }\n\n        Ok(())\n    }\n\n    /// Lists all Parquet files in a specified directory.\n    ///\n    /// This method scans a directory and returns the full paths of all files with the `.parquet`\n    /// extension. It works with both local filesystems and remote object stores, making it\n    /// suitable for various storage backends.\n    ///\n    /// # Parameters\n    ///\n    /// - `directory`: The directory path to scan for Parquet files.\n    ///\n    /// # Returns\n    ///\n    /// Returns a vector of full file paths (as strings) for all Parquet files found in the directory.","sourceCodeStart":1270,"sourceCodeEnd":1306,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/persistence/src/backend/catalog.rs#L1270-L1306","documentation":"`extend_file_name` merges a source file into a target file within a directory and then re-validates that the directory's file intervals are still disjoint. If the extended file's new interval overlaps a neighbor's, the operation is rolled back semantically and the error is raised, since the post-condition invariant was violated.","triggerScenarios":"Calling `extend_file_name` with a source file whose time range spills outside the target file's interval into an adjacent file's range — e.g. extending a file with data that belongs to the next segment.","commonSituations":"Manual file consolidation/cleanup scripts picking the wrong target file; merging segments after deleting an intermediate file; automated compaction that mis-assigned boundaries.","solutions":["Choose a target file whose interval fully contains the source file's interval.","Check directory intervals before extending and pick the correct neighbor.","Split the source file and extend each part into its appropriate target.","If overlaps are intentional, restructure the directory (rewrite all files) rather than extending."],"exampleFix":"// before\ncatalog.extend_file_name(dir, source_file, target_file)?;\n// after\nlet intervals = catalog.get_directory_intervals(dir)?;\nif contains(intervals, source_range, target_file) {\n    catalog.extend_file_name(dir, source_file, target_file)?;\n} else {\n    // pick the file whose interval contains source_range, or split the source\n}","handlingStrategy":"validation","validationCode":"let intervals = catalog.get_directory_intervals(&dir)?;\n// only extend when the source range is contained by the target's interval\nassert!(intervals.iter().any(|iv| contains(iv, source_range)),\n    \"source range not contained by any existing interval\");","typeGuard":null,"tryCatchPattern":"match catalog.extend_file_name(&dir, &source, &target) {\n    Err(e) if e.to_string().contains(\"not disjoint after extending\") => {\n        // wrong target: pick the file whose interval contains the source range\n        let target2 = pick_containing_file(&dir, &source)?;\n        catalog.extend_file_name(&dir, &source, &target2)?;\n    }\n    other => other?,\n}","preventionTips":["Inspect directory intervals before choosing an extend target","Only extend a file with data inside its own time range","Do full rewrites instead of extends when boundaries are uncertain","Back up the directory before manual compaction scripts"],"tags":["rust","persistence","parquet","intervals","file-management"],"backgroundTag":"internal-invariant-violation","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}