{"record":{"id":"e9c19c9902ae3f79","repo":"hashicorp/nomad","slug":"failed-to-load-si-token-for-s-w","errorCode":null,"errorMessage":"failed to load SI token for %s: %w","messagePattern":"failed to load SI token for (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/taskrunner/envoy_bootstrap_hook.go","lineNumber":635,"sourceCode":"\t\tenv = append(env, fmt.Sprintf(\"%s=%s\", \"CONSUL_HTTP_SSL_VERIFY\", v))\n\t}\n\tif v := e.namespace; v != \"\" {\n\t\tenv = append(env, fmt.Sprintf(\"%s=%s\", \"CONSUL_NAMESPACE\", v))\n\t}\n\treturn env\n}\n\n// maybeLoadSIToken reads the SI token saved to disk in the secrets directory\n// by the service identities prestart hook. This envoy bootstrap hook blocks\n// until the sids hook completes, so if the SI token is required to exist (i.e.\n// Consul ACLs are enabled), it will be in place by the time we try to read it.\nfunc (h *envoyBootstrapHook) maybeLoadSIToken(task, dir string) (string, error) {\n\ttokenPath := filepath.Join(dir, sidsTokenFile)\n\ttoken, err := os.ReadFile(tokenPath)\n\tif err != nil {\n\t\tif !os.IsNotExist(err) {\n\t\t\th.logger.Error(\"failed to load SI token\", \"task\", task, \"error\", err)\n\t\t\treturn \"\", fmt.Errorf(\"failed to load SI token for %s: %w\", task, err)\n\t\t}\n\t\th.logger.Trace(\"no SI token to load, falling back to agent token\", \"task\", task)\n\t\treturn h.consulFallbackToken, nil // token file does not exist\n\t}\n\th.logger.Trace(\"recovered pre-existing SI token\", \"task\", task)\n\treturn string(token), nil\n}\n\nfunc (h *envoyBootstrapHook) servicePreflightCheck(\n\tctx context.Context, backoffOpts decay.BackoffOptions, proxyServiceID string) error {\n\n\t// keep track of latest error returned from Consul or from missing service\n\tvar apiErr error\n\tvar allocServices *serviceregistration.AllocRegistration\n\n\tbackoffErr := decay.Backoff(func() (bool, error) {\n\t\t// If hook is killed, just stop.\n\t\tselect {","sourceCodeStart":617,"sourceCodeEnd":653,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/taskrunner/envoy_bootstrap_hook.go#L617-L653","documentation":"maybeLoadSIToken reads the Consul SI (service identity) JWT token file from the task's secrets directory. If os.ReadFile fails with an error other than NotExist, the hook logs it and wraps the error with the task name. A missing file is tolerated (falls back to the Consul agent's token), but any other read error is fatal to Prestart.","triggerScenarios":"os.ReadFile(filepath.Join(dir, sidsTokenFile)) returns a permission error, an I/O error, or the path exists but is a directory — anything other than os.IsNotExist.","commonSituations":"SI token file created by a previous hook has wrong permissions or ownership; secrets dir on a failing filesystem; file replaced by a directory due to a misbehaving template/other hook.","solutions":["Check ownership/permissions of secrets/ and the SI token file so the Nomad client (task) user can read it.","Verify the file at that path is a regular file, not a directory or symlink to nowhere.","Check alloc-dir filesystem health (dmesg, mount status).","If your workload doesn't need workload identity, ensure the SI token file is simply absent (that path is handled gracefully) rather than malformed."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"tokenPath := filepath.Join(secretsDir, \"consul_identity\")\nif fi, err := os.Stat(tokenPath); err != nil && !os.IsNotExist(err) {\n    return fmt.Errorf(\"SI token file unusable: %w\", err)\n} else if fi != nil && !fi.Mode().IsRegular() {\n    return fmt.Errorf(\"SI token path is not a regular file\")\n}","typeGuard":null,"tryCatchPattern":"token, err := os.ReadFile(tokenPath)\nswitch {\ncase err == nil:\n    useToken(string(token))\ncase os.IsNotExist(err):\n    useToken(fallbackAgentToken) // tolerated by the hook\ndefault:\n    log.Error(\"SI token read failed\", \"error\", err) // check perms/IO\n}","preventionTips":["Let Nomad manage SI token file lifecycle; don't rewrite its permissions externally.","Verify secrets dir ownership matches the Nomad client user.","Check template hooks aren't replacing the token file with a directory.","Alert on filesystem read errors on the alloc-dir volume."],"tags":["go","filesystem","nomad","consul","si-token"],"backgroundTag":"si-token-read-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}