{"record":{"id":"33358dcc385096fd","repo":"hashicorp/nomad","slug":"failed-to-create-keystore-tempdir","errorCode":null,"errorMessage":"Failed to create keystore tempdir","messagePattern":"Failed to create keystore tempdir","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/server.go","lineNumber":436,"sourceCode":"\t// Create the node heartbeater\n\ts.nodeHeartbeater = newNodeHeartbeater(s)\n\n\t// Create the periodic dispatcher for launching periodic jobs.\n\ts.periodicDispatcher = NewPeriodicDispatch(s.logger, s)\n\n\t// Initialize the stats fetcher that autopilot will use.\n\ts.statsFetcher = NewStatsFetcher(s.logger, s.connPool, s.config.Region)\n\ts.statsFetcher.SetLocalServer(s)\n\n\t// Setup Consul\n\ts.consulConfigEntries = NewConsulConfigsAPI(consulConfigFunc, s.logger)\n\n\t// Set up the keyring\n\tkeystorePath := filepath.Join(s.config.DataDir, \"keystore\")\n\tif s.config.DevMode && s.config.DataDir == \"\" {\n\t\tkeystorePath, err = os.MkdirTemp(\"\", \"nomad-keystore\")\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"Failed to create keystore tempdir\")\n\t\t}\n\t}\n\tencrypter, err := NewEncrypter(s, keystorePath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\ts.encrypter = encrypter\n\n\t// Set up the OIDC discovery configuration required by third parties, such as\n\t// AWS's IAM OIDC Provider, to authenticate workload identity JWTs.\n\tif iss := config.OIDCIssuer; iss != \"\" {\n\t\toidcDisco, err := structs.NewOIDCDiscoveryConfig(iss)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\ts.oidcDisco = oidcDisco\n\t\ts.logger.Info(\"issuer set; OIDC Discovery endpoint for workload identities enabled\", \"issuer\", iss)\n\t} else {","sourceCodeStart":418,"sourceCodeEnd":454,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/server.go#L418-L454","documentation":"During server startup NewServer sets up the encryption keyring. In dev mode with no DataDir, it creates a temporary directory for the keystore via os.MkdirTemp; if that OS call fails, the server returns this error. Notably the error discards the underlying cause, so only the OS-level reason (permissions, /tmp full, TMPDIR invalid) explains it.","triggerScenarios":"Starting a Nomad server with dev_mode=true and DataDir==\"\" while os.MkdirTemp(\"\", \"nomad-keystore\") fails — e.g. TMPDIR points to a nonexistent/unwritable path or the filesystem is full.","commonSituations":"Containers with read-only or missing temp directories; TMPDIR misconfigured in systemd units or Docker; disk exhaustion on the node running dev-mode servers or local test setups.","solutions":["Fix TMPDIR or ensure the default temp location exists and is writable, then restart","Set an explicit data_dir so the keystore uses a real path instead of a temp dir","Free disk space if the temp filesystem is full","Inspect the OS error with `TMPDIR=/path nomad agent -dev` to confirm the cause"],"exampleFix":"// before (fails when TMPDIR unwritable)\nnomad agent -dev\n// after\nnomad agent -dev -data-dir /var/lib/nomad","handlingStrategy":"validation","validationCode":"// Ensure a writable temp location before starting a dev-mode server\nif cfg.DevMode && cfg.DataDir == \"\" {\n    if d, err := os.MkdirTemp(\"\", \"nomad-keystore-test\"); err != nil {\n        return fmt.Errorf(\"temp dir unavailable: %w\", err)\n    } else { os.RemoveAll(d) }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always set an explicit data_dir, even in dev, to avoid temp-dir dependence","Verify TMPDIR exists and is writable in containers/systemd units","Monitor disk space on the temp filesystem"],"tags":["nomad","go","filesystem","keystore","dev-mode"],"backgroundTag":"temp-dir-creation-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}