vectordotdev/vector · critical

effective reader file ID must be in the checkpoint window

Error message

effective reader file ID must be in the checkpoint window

What it means

During startup, Vector's v2 disk buffer replays its on-disk ledger to locate the reader's position within the checkpointed window of data-file IDs. After `find_checkpoint_reader_file_id` resolves an effective reader file ID, the code expect()s that this ID actually appears in `checkpoint_data_file_ids`. If the persisted ledger/checkpoint state is self-inconsistent (reader file not among the checkpointed files), this invariant panic aborts buffer construction and Vector crash-loops until the on-disk state is removed.

Source

Thrown at lib/vector-buffers/src/variants/disk_v2/checkpoint_recovery.rs:294

        let checkpoint_data_file_ids = self
            .checkpoint_data_file_ids(reader_file_id, writer_file_id)
            .await?;
        // The ledger's reader file ID is only the durable file checkpoint. It can lag behind the
        // durable record checkpoint when the reader acknowledged records in later files but crashed
        // before flushing the advanced file ID, so re-evaluate the effective reader boundary from
        // `reader_last_record_id`.
        let effective_reader_file_id = self
            .find_checkpoint_reader_file_id(
                &checkpoint_data_file_ids,
                writer_file_id,
                reader_last_record_id,
            )
            .await?;
        let mut unread_buffer_size = 0;
        let effective_reader_position = checkpoint_data_file_ids
            .iter()
            .position(|&data_file_id| data_file_id == effective_reader_file_id)
            .expect("effective reader file ID must be in the checkpoint window");

        for &data_file_id in &checkpoint_data_file_ids[effective_reader_position..] {
            let data_file_path = self.ledger().get_data_file_path(data_file_id);
            let boundary_file = CheckpointBoundaryFile {
                id: data_file_id,
                path: &data_file_path,
            };
            unread_buffer_size += if effective_reader_file_id == writer_file_id {
                let scan = CheckpointBoundaryScan::reader_writer(
                    boundary_file,
                    reader_last_record_id,
                    writer_next_record_id,
                );
                let scan = self.reconcile_checkpoint_boundary_data_file(scan).await?;
                scan.recovered_bytes()
            } else if data_file_id == effective_reader_file_id {
                let scan = CheckpointBoundaryScan::reader(boundary_file, reader_last_record_id);
                let scan = self.reconcile_checkpoint_boundary_data_file(scan).await?;

View on GitHub (pinned to 3708c39b12)

Solutions

1. Stop Vector and move the buffer's data directory aside so it is recreated empty (buffered events are lost), then restart
2. Ensure exactly one Vector instance owns each data directory and that no other tooling touches its files
3. Run a single Vector version across restarts; never downgrade a disk_v2 data directory
4. If it reproduces on a clean directory with one instance, preserve the ledger/data files and file a Vector bug

When it happens

Trigger: Building a `disk` buffer (type = "disk_v2") whose data directory holds a ledger/checkpoint written inconsistently: two Vector processes sharing one data directory, unclean shutdown mid-checkpoint flush, disk corruption, or reusing a data directory across Vector versions with a changed on-disk format.

Common situations: Two Vector containers pointed at the same `data_directory`; host power loss or SIGKILL during heavy buffered writes; downgrading Vector after a disk_v2 format change; bit rot or a full disk truncating the ledger.

Related errors

• Reader encountered unrecoverable error: {e:?}
• record was already validated
• Event count for a record cannot exceed 2^64 events.
• a record with a next ID must have an event count
• event count should never exceed u64

AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/18d0979b46707c4c. Report an issue: GitHub.