hashicorp/nomad · error

failed to create secrets/envoy_bootstrap.json for envoy: %w

Error message

failed to create secrets/envoy_bootstrap.json for envoy: %w

What it means

Inside the envoy bootstrap retry closure, the hook runs `consul envoy bootstrap` and redirects its stdout into secrets/envoy_bootstrap.json via os.Create. This error wraps a failure of os.Create, i.e. Nomad could not create/truncate the bootstrap output file, so the command was never run.

Source

Thrown at client/allocrunner/taskrunner/envoy_bootstrap_hook.go:374

	// Since Consul services are registered asynchronously with this task
	// hook running, retry until timeout or success.
	backoffErr := decay.Backoff(func() (bool, error) {
		// If hook is killed, just stop.
		select {
		case <-ctx.Done():
			return false, nil
		default:
		}

		// Prepare bootstrap command to run.
		cmd := exec.CommandContext(ctx, "consul", bootstrapArgs...)
		cmd.Env = bootstrapEnv

		// Redirect stdout to secrets/envoy_bootstrap.json.
		stdout, fileErr := os.Create(bootstrapFilePath)
		if fileErr != nil {
			return false, fmt.Errorf("failed to create secrets/envoy_bootstrap.json for envoy: %w", fileErr)
		}
		defer stdout.Close()
		cmd.Stdout = stdout

		// Redirect stderr into another file for later debugging.
		stderr, fileErr := os.OpenFile(bootstrapStderrPath, os.O_RDWR|os.O_CREATE, 0644)
		if fileErr != nil {
			return false, fmt.Errorf("failed to create alloc/logs/envoy_bootstrap.stderr.0 for envoy: %w", fileErr)
		}
		defer stderr.Close()
		cmd.Stderr = stderr

		// Generate bootstrap
		cmdErr = cmd.Run()

		// Command succeeded, exit.
		if cmdErr == nil {
			// Bootstrap written. Move on without marking as Done as Prestart needs

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Verify the alloc dir secrets/ directory exists and is writable by the Nomad client user.
  2. Check host disk space and mount health (dmesg for I/O errors).
  3. Check SELinux/AppArmor logs if file creation is blocked by policy.
  4. Restart the allocation to re-run Prestart and recreate the directory tree.
Defensive patterns

Strategy: validation

Validate before calling

// before running the hook, confirm the target directory is creatable/writable
if err := os.MkdirAll(filepath.Join(allocDir, "secrets"), 0o750); err != nil { return err }
probe, err := os.Create(filepath.Join(allocDir, "secrets", ".writecheck"))
if err != nil { return err }
probe.Close()
os.Remove(probe.Name())

Prevention

When it happens

Trigger: os.Create(bootstrapFilePath) fails in the retry closure — the secrets directory doesn't exist, the file can't be created due to permissions, or a filesystem/I/O error occurs.

Common situations: The alloc dir's secrets/ directory was removed or never created; wrong ownership/permissions on the task dir; read-only or full filesystem; SELinux/AppArmor denying file creation.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/d17a46c57e7ebf67. Report an issue: GitHub.