nautechsystems/nautilus_trader · error · anyhow::Error

Intervals are not contiguous. When ensure_contiguous_files=t

Error message

Intervals are not contiguous. When ensure_contiguous_files=true, all files in the consolidation range must have contiguous timestamps.

What it means

Raised by prepare_consolidation_queries when consolidation is requested with ensure_contiguous_files=true but the Parquet files (intervals) in the target range have gaps between their timestamps. The library requires that consolidation only merge files whose intervals form a contiguous sequence, otherwise the merged file would misrepresent data availability. This is a data-integrity guard for the catalog's file layout.

Source

Thrown at crates/persistence/src/backend/catalog_operations.rs:1282

        let mut filtered_intervals = Vec::new();

        for &(interval_start, interval_end) in intervals {
            // Check if interval overlaps with the specified range
            if used_start.is_none_or(|used_start| used_start <= interval_end)
                && used_end.is_none_or(|used_end| interval_start <= used_end)
            {
                filtered_intervals.push((interval_start, interval_end));
            }
        }

        if filtered_intervals.is_empty() {
            return Ok(Vec::new()); // No intervals in the specified range
        }

        // Check contiguity of filtered intervals if required
        if ensure_contiguous_files && !are_intervals_contiguous(&filtered_intervals) {
            anyhow::bail!(
                "Intervals are not contiguous. When ensure_contiguous_files=true, \
                 all files in the consolidation range must have contiguous timestamps."
            );
        }

        // Group intervals by the target period: split only when the gap between files
        // exceeds one period, since sub-period gaps land in the same consolidated file.
        let contiguous_groups = self.group_contiguous_intervals(&filtered_intervals, period_nanos);

        let mut queries_to_execute = Vec::new();

        // Handle interval splitting by creating split operations for data preservation
        if !filtered_intervals.is_empty() {
            if let Some(start_ts) = used_start {
                let first_interval = filtered_intervals[0];
                if first_interval.0 < start_ts && start_ts <= first_interval.1 {
                    // Split before start: preserve data from interval_start to start-1
                    queries_to_execute.push(ConsolidationQuery {

View on GitHub (pinned to 18893faf8b)

Solutions

1. Set ensure_contiguous_files=false if gaps are acceptable and consolidation need not preserve contiguity.
2. Narrow the consolidation range so it only includes files with contiguous timestamps.
3. Backfill or regenerate the missing data files so intervals become contiguous, then re-run consolidation.
4. Inspect the catalog intervals (e.g. via get_directory_intervals) to identify which files have gaps before retrying.

Example fix

// before
let queries = catalog.prepare_consolidation_queries(
    &identifier, data_type, start, end, /* ensure_contiguous_files */ true,
)?;
// after — allow gaps in the range
let queries = catalog.prepare_consolidation_queries(
    &identifier, data_type, start, end, /* ensure_contiguous_files */ false,
)?;

// check contiguity before requesting consolidation
let intervals = catalog.get_directory_intervals(&directory)?;
if ensure_contiguous && !are_intervals_contiguous(&intervals) {
    // narrow the range or backfill gaps first
    return Err("range has gaps; consolidation would produce non-contiguous file".into());
}

match catalog.consolidate_data_by_period_generic(...) {
    Err(e) if e.to_string().contains("Intervals are not contiguous") => {
        // retry with ensure_contiguous_files=false or backfill missing files
    }
    other => other?,
}

When it happens

Trigger: Calling consolidate_data_by_period_generic or consolidate_custom_data_by_period with ensure_contiguous_files=true while the range contains files with non-contiguous start/end timestamps (e.g. missing sessions, deleted files in the middle of the range).

Common situations: Ranges spanning holidays/weekends where no data was recorded, files deleted individually for backfill or storage reclamation, ingestion that skipped periods due to feed outages, or manually copied catalog directories with partial data.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Intervals are not disjoint after resetting file names
• Replacement hash {transaction_hash} conflicts with another i
• Included wrap transaction {tx_hash} has invalid block number
• WETH balance overflow for included transaction {tx_hash} at
• conflicting RTDS TWAP observation topic={} symbol={} timesta

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/9d9392345f1b64f2. Report an issue: GitHub.