hashicorp/nomad · error

Failed to query list of containers: %s

Error message

Failed to query list of containers: %s

What it means

containerByName lists Docker containers via ContainerList to find a container matching a Nomad-generated name. If the Docker API list call fails (daemon unreachable, timeout, permissions), the error is wrapped with recoverableErrTimeouts and returned to createContainer/CreateNetwork.

Source

Thrown at drivers/docker/driver.go:1695

	}

	return ip, auto
}

// containerByName finds a running container by name, and returns an error
// if the container is dead or can't be found.
func (d *Driver) containerByName(name string) (*mclient.ContainerInspectResult, error) {

	dockerClient, err := d.getDockerClient()
	if err != nil {
		return nil, err
	}
	containers, err := dockerClient.ContainerList(d.ctx, mclient.ContainerListOptions{All: true})
	if err != nil {
		d.logger.Error("failed to query list of containers matching name",
			"container_name", name)
		return nil, recoverableErrTimeouts(
			fmt.Errorf("Failed to query list of containers: %s", err))
	}

	// container names with a / pre-pended to the Nomad generated container names
	containerName := "/" + name
	var (
		shimContainer containerapi.Summary
		found         bool
	)
OUTER:
	for _, shimContainer = range containers.Items {
		d.logger.Trace("listed container", "names", hclog.Fmt("%+v", shimContainer.Names))
		for _, name := range shimContainer.Names {
			if name == containerName {
				d.logger.Trace("Found container",
					"container_name", containerName, "container_id", shimContainer.ID)
				found = true
				break OUTER
			}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check `docker ps` and `systemctl status docker` on the client host to confirm the daemon is healthy.
2. Retry the allocation — the error is wrapped as recoverable for timeouts, so rescheduling often succeeds once the daemon recovers.
3. Verify docker.sock permissions for the nomad user and DOCKER_HOST/socket configuration in the client's docker plugin config.

Example fix

// before
sudo docker ps   # fails: Cannot connect to the Docker daemon
// after
sudo systemctl restart docker
# then reschedule: nomad job eval <job-id> or nomad alloc stop <alloc-id>

// preflight: confirm the daemon answers before scheduling
cmd := exec.Command("docker", "info")
if err := cmd.Run(); err != nil { log.Fatal("docker daemon unreachable") }

if err != nil {
    if errors.Is(err, context.DeadlineExceeded) || isTimeoutErr(err) {
        time.Sleep(backoff); goto retry // recoverable per recoverableErrTimeouts
    }
    return err
}

When it happens

Trigger: Docker daemon down or restarting, socket permissions denied, API timeout under load, or network partition between Nomad and the Docker endpoint during createContainer or CreateNetwork.

Common situations: Docker service crash/OOM on the host, docker.sock permissions changed, Docker API rate limiting or slowness with many containers, upgrade of the Docker daemon mid-allocation.

Related errors

• failed to inspect container state: %v
• running container as ContainerAdmin is unsafe; change the co
• error decoding stats data: no reader body
• error decoding stats data: stats were nil
• does not match registry specification

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/7c4013a646837377. Report an issue: GitHub.