{"record":{"id":"e1aa57d207fe27f0","repo":"risingwavelabs/risingwave","slug":"failed-to-get-secret-in-secret-manager-secret-id","errorCode":null,"errorMessage":"Failed to get secret in secret manager, secret_id: {}","messagePattern":"Failed to get secret in secret manager, secret_id: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/frontend/src/handler/alter_secret.rs","lineNumber":46,"sourceCode":"\npub async fn handle_alter_secret(\n    handler_args: HandlerArgs,\n    secret_name: ObjectName,\n    sql_options: Vec<SqlOption>,\n    credential: Value,\n) -> Result<RwPgResponse> {\n    Feature::SecretManagement.check_available()?;\n\n    let session = handler_args.session;\n\n    if let Some((secret_catalog, _, _)) =\n        fetch_secret_catalog_with_db_schema_id(&session, &secret_name, false)?\n    {\n        let secret_id = secret_catalog.id;\n        let secret_payload = if sql_options.is_empty() {\n            let original_pb_secret_bytes = LocalSecretManager::global()\n                .get_secret(secret_id)\n                .ok_or(anyhow!(\n                    \"Failed to get secret in secret manager, secret_id: {}\",\n                    secret_id\n                ))?;\n            let original_secret_backend =\n                LocalSecretManager::get_pb_secret_backend(&original_pb_secret_bytes)?;\n            match original_secret_backend {\n                secret::SecretBackend::Meta(_) => {\n                    let new_secret_value_bytes = secret_to_str(&credential)?.as_bytes().to_vec();\n                    let secret_payload = risingwave_pb::secret::Secret {\n                        secret_backend: Some(risingwave_pb::secret::secret::SecretBackend::Meta(\n                            risingwave_pb::secret::SecretMetaBackend {\n                                value: new_secret_value_bytes,\n                            },\n                        )),\n                    };\n                    secret_payload.encode_to_vec()\n                }\n                secret::SecretBackend::HashicorpVault(_vault_backend) => {","sourceCodeStart":28,"sourceCodeEnd":64,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/frontend/src/handler/alter_secret.rs#L28-L64","documentation":"Raised in `handle_alter_secret` when ALTER SECRET needs the original secret payload (to preserve unchanged fields like the backend type) but `LocalSecretManager::global().get_secret(secret_id)` returns `None`. It means the in-memory local secret manager in the frontend process does not hold the secret with the given id, even though the secret catalog entry exists.","triggerScenarios":"Running `ALTER SECRET name ...` without new payload options (empty `sql_options`), which forces a read of the existing secret from the local secret manager, when the frontend has not loaded/cached that secret (e.g. after a frontend restart, a stale meta cache, or when the secret was created by a different node and not synced).","commonSituations":"Frontend service restarted and local secret cache is empty or stale; ALTER issued on a node different from where the secret was materialized; meta/frontend version skew or cache invalidation problems after failover.","solutions":["Provide the full secret payload in the ALTER statement (include new content options) so the original secret does not need to be fetched from the local manager.","Restart the frontend/meta services to force re-sync of secrets from the meta store, then retry the ALTER.","Check meta node logs for secret sync/cache errors and verify the secret exists via `SHOW SECRETS` / system catalog.","Upgrade to a version where secret cache sync between meta and frontend is fixed if this reproduces after failover."],"exampleFix":"-- before (forces read of cached original payload)\nALTER SECRET my_secret;\n-- after (supply payload so local cache lookup is avoided)\nALTER SECRET my_secret WITH (password = 'new_value');","handlingStrategy":"fallback","validationCode":"// before ALTER without new payload, confirm the secret is resolvable\nSHOW SECRETS; -- ensure the target secret exists and consider always supplying the payload","typeGuard":null,"tryCatchPattern":"match LocalSecretManager::global().get_secret(secret_id) {\n    Some(bytes) => proceed_with_original(bytes),\n    None => {\n        // fallback: require caller-supplied payload or trigger meta resync, then retry once\n        return Err(anyhow!(\"secret {} not cached locally; re-run ALTER with full payload\", secret_id));\n    }\n}","preventionTips":["Always pass the complete secret payload in ALTER SECRET instead of relying on the cached original.","Avoid ALTER operations immediately after a frontend restart before caches warm up.","Monitor secret sync between meta and frontend nodes; alert on cache misses.","Keep frontend and meta nodes on matching versions."],"tags":["secrets","cache","alter","risingwave"],"backgroundTag":"record-not-found","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}