{"record":{"id":"46689f5056b3676d","repo":"clockworklabs/SpacetimeDB","slug":"unable-to-read-private-key-for-jwt-token-signing","errorCode":null,"errorMessage":"Unable to read private key for JWT token signing","messagePattern":"Unable to read private key for JWT token signing","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"critical","filePath":"crates/core/src/auth/mod.rs","lineNumber":66,"sourceCode":"// Get the key pair if the given files exist. If they don't, create them.\n// If only one of the files exists, return an error.\npub fn get_or_create_keys(certs: &CertificateAuthority) -> anyhow::Result<JwtKeys> {\n    let public_key_path = &certs.jwt_pub_key_path;\n    let private_key_path = &certs.jwt_priv_key_path;\n\n    let public_key_bytes = public_key_path.read().ok();\n    let private_key_bytes = private_key_path.read().ok();\n\n    // If both keys are unspecified, create them\n    let key_pair = match (public_key_bytes, private_key_bytes) {\n        (Some(pub_), Some(priv_)) => EcKeyPair::new(pub_, priv_),\n        (None, None) => {\n            let keys = EcKeyPair::generate()?;\n            keys.write_to_files(public_key_path, private_key_path)?;\n            keys\n        }\n        (None, Some(_)) => anyhow::bail!(\"Unable to read public key for JWT token verification\"),\n        (Some(_), None) => anyhow::bail!(\"Unable to read private key for JWT token signing\"),\n    };\n\n    key_pair.try_into()\n}\n\n// An Ec key pair in pem format.\npub struct EcKeyPair {\n    pub public_key_bytes: Vec<u8>,\n    pub private_key_bytes: Vec<u8>,\n}\n\nimpl TryFrom<EcKeyPair> for JwtKeys {\n    type Error = anyhow::Error;\n    fn try_from(pair: EcKeyPair) -> anyhow::Result<Self> {\n        JwtKeys::new(pair.public_key_bytes, pair.private_key_bytes)\n    }\n}\n","sourceCodeStart":48,"sourceCodeEnd":84,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/6dee26c6efc2856793e12b148a59742964f5d783/crates/core/src/auth/mod.rs#L48-L84","documentation":"On startup the node loads the EC key pair used to sign JWTs. If the public key file loads but the private key file cannot be read (missing, unreadable, or corrupt), startup aborts with this message. The node will not regenerate just one half of the pair -- keys are only auto-created when both files are absent, so a missing private key must be restored or the whole pair regenerated.","triggerScenarios":"The public key file exists but the private key file was deleted, moved, has a 0-byte/truncated body, or permissions deny the node process read access; a backup that captured only the public half; key paths misconfigured so they point at different directories.","commonSituations":"Secrets scrubbing (e.g. deploy pipelines or .gitignore rules) removing the private key but leaving the public one; a failed key rotation; Docker secrets mounted incorrectly; running under a different user than the key files' owner.","solutions":["Restore the matching private key from your secret backup (it must correspond to the existing public key)","Otherwise regenerate the pair: delete BOTH key files and restart the node -- a fresh pair is generated and persisted (all previously issued JWTs become invalid)","Fix permissions/ownership so the node process can read the private key","Verify the pair matches: `openssl pkey -in private.pem -pubout` vs the public key file"],"exampleFix":"# before: only the public key is present\n$ ls /keys\njwt_public.pem\n\n# after: restore the matching private key, or regenerate the whole pair\n$ cp /backup/jwt_private.pem /keys/ && chown spacetimedb /keys/jwt_private.pem && systemctl restart spacetimedb","handlingStrategy":"validation","validationCode":"#!/usr/bin/env bash\n# run before starting the node\nfor f in \"$JWT_PUBLIC_KEY\" \"$JWT_PRIVATE_KEY\"; do\n  [ -r \"$f\" ] && [ -s \"$f\" ] || { echo \"missing/empty key: $f\" >&2; exit 1; }\ndone\nopenssl pkey -in \"$JWT_PRIVATE_KEY\" -pubout 2>/dev/null | diff -q - \"$JWT_PUBLIC_KEY\" \\\n  || echo 'WARNING: key pair mismatch' >&2","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Store the private key in a proper secret manager; make deploys verify it exists before cutover","Never let cleanup jobs/gitignore rules delete one half of the pair","Verify pair coherence (openssl pkey -pubout) in a pre-start check","If regenerating, remove both files and plan for re-issuing tokens"],"tags":["auth","jwt","keys","server-startup","secrets"],"backgroundTag":"key-pair-missing-or-corrupt","analyzedSha":"6dee26c6efc2856793e12b148a59742964f5d783","analyzedAt":"2026-08-20T06:08:37.179Z","contentChangedAt":"2026-08-20T06:08:37.179Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}