vectordotdev/vector · error

Every started ContainerId has it's ContainerState

Error message

Every started ContainerId has it's ContainerState

What it means

In the docker_logs actor's error branch, a failed log stream reports (id, ErrorPersistence); the actor removes its state with containers.remove(&id).expect("Every started ContainerId has it's ContainerState"). The invariant: an error can only arrive for a container the actor started and still tracks. The panic fires when an error arrives for an id that was already removed or never inserted - the same bookkeeping desync as the info path, seen from the failure side.

Source

Thrown at src/sources/docker_logs/mod.rs:606

    async fn run(mut self) {
        loop {
            tokio::select! {
                value = self.main_recv.recv() => {
                    match value {
                        Some(Ok(info)) => {
                            let state = self
                                .containers
                                .get_mut(&info.id)
                                .expect("Every ContainerLogInfo has it's ContainerState");
                            if state.return_info(info) {
                                self.esb.restart(state);
                            }
                        },
                        Some(Err((id,persistence))) => {
                            let state = self
                                .containers
                                .remove(&id)
                                .expect("Every started ContainerId has it's ContainerState");
                            match persistence{
                                ErrorPersistence::Transient => if state.is_running() {
                                    let backoff= Some(self.backoff_duration);
                                    self.containers.insert(id.clone(), self.esb.start(id, backoff));
                                }
                                // Forget the container since the error is permanent.
                                ErrorPersistence::Permanent => (),
                            }
                        }
                        None => {
                            error!(message = "The docker_logs source main stream has ended unexpectedly.", internal_log_rate_limit = false);
                            info!(message = "Shutting down docker_logs source.");
                            return;
                        }
                    };
                }
                value = self.events.next() => {
                    match value {

View on GitHub (pinned to 3708c39b12)

Solutions

1. Gather logs (Vector version, docker events, debug output) and report upstream as an internal invariant break
2. Upgrade/downgrade to a docker_logs-stable release
3. Patch: use containers.remove(&id) and warn-and-continue on None instead of expecting
4. Supervise and restart Vector on this panic until a fixed build is deployed

Example fix

// before
let state = self
    .containers
    .remove(&id)
    .expect("Every started ContainerId has it's ContainerState");

// after
let Some(state) = self.containers.remove(&id) else {
    warn!(message = "error reported for unknown container, skipping", id = %id);
    continue;
};

// before removing in the error branch:
if let Some(state) = self.containers.remove(&id) {
    // existing ErrorPersistence handling
} else {
    warn!(message = "error for untracked container", id = %id);
    continue;
}

fn remove_state(
    containers: &mut HashMap<ContainerId, ContainerState>,
    id: &ContainerId,
) -> Option<ContainerState> {
    containers.remove(id)
}

match self.containers.remove(&id) {
    Some(state) => match persistence {
        ErrorPersistence::Transient => { /* restart */ }
        ErrorPersistence::Permanent => (),
    },
    None => warn!(id = %id, "duplicate or unknown container error"),
}

When it happens

Trigger: An error message for a container whose state was already removed (double-report of a failed stream, or a race between return_info and the error report for the same id).

Common situations: Docker daemon restart storms, aggressive container churn, or docker_logs refactors/regressions in a given Vector version.

Related errors

• Every ContainerLogInfo has it's ContainerState
• `.` must exist in the event
• `message` must exist in the event
• `docker_logs.stream` must exist in the metadata
• stream must exist in the event

AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/c65a6d965df20c04. Report an issue: GitHub.