{"record":{"id":"7e63cdbc2d91f934","repo":"neondatabase/neon","slug":"cannot-manage-failpoints-because-neon-was-compiled","errorCode":null,"errorMessage":"Cannot manage failpoints because neon was compiled without failpoints support","messagePattern":"Cannot manage failpoints because neon was compiled without failpoints support","errorType":"http","errorClass":"ApiError","httpStatus":400,"severity":"warning","filePath":"libs/http-utils/src/failpoints.rs","lineNumber":28,"sourceCode":"\n/// Information for configuring a single fail point\n#[derive(Debug, Serialize, Deserialize)]\npub struct FailpointConfig {\n /// Name of the fail point\n pub name: String,\n /// List of actions to take, using the format described in `fail::cfg`\n ///\n /// We also support `actions = \"exit\"` to cause the fail point to immediately exit.\n pub actions: String,\n}\n\n/// Configure failpoints through http.\npub async fn failpoints_handler(\n mut request: Request,\n _cancel: CancellationToken,\n) -> Result, ApiError> {\n if !fail::has_failpoints() {\n return Err(ApiError::BadRequest(anyhow::anyhow!(\n \"Cannot manage failpoints because neon was compiled without failpoints support\"\n )));\n }\n\n let failpoints: ConfigureFailpointsRequest = json_request(&mut request).await?;\n for fp in failpoints {\n tracing::info!(\"cfg failpoint: {} {}\", fp.name, fp.actions);\n\n // We recognize one extra \"action\" that's not natively recognized\n // by the failpoints crate: exit, to immediately kill the process\n let cfg_result = apply_failpoint(&fp.name, &fp.actions);\n\n if let Err(err_msg) = cfg_result {\n return Err(ApiError::BadRequest(anyhow::anyhow!(\n \"Failed to configure failpoints: {err_msg}\"\n )));\n }\n }","sourceCodeStart":10,"sourceCodeEnd":46,"githubUrl":"https://github.com/neondatabase/neon/blob/8f60b04da47ffefe0e52bda2440134b42874eb75/libs/http-utils/src/failpoints.rs#L10-L46","documentation":"Returned as HTTP 400 BadRequest by failpoints_handler in neon's http-utils when the binary was compiled without failpoints support: fail::has_failpoints() is false because the failpoints crate was compiled with its no-op configuration. The /failpoints management endpoint exists in every build, but only builds with the failpoints cargo feature actually embed failpoint instrumentation, so test/chaos scripts hitting the endpoint on a production build get this error.","triggerScenarios":"POST /failpoints with a ConfigureFailpointsRequest body against a neon service binary compiled without --features failpoints; the handler rejects the request before parsing or applying any failpoint configuration.","commonSituations":"Running chaos/test scripts designed for CI builds against locally built or release binaries; forgetting to enable the failpoints feature in a custom build; CI image rebuilds that dropped the feature flag.","solutions":["Rebuild the target service with failpoints enabled, e.g. cargo build --features failpoints (or the neon-specific testing profile that turns it on)","Point the failpoint test at a binary that was compiled with the feature (CI's test builds always are)","If you cannot rebuild, remove the failpoint-configuration step from the scenario"],"exampleFix":"# before\ncargo build -p neon_pageserver # no failpoints feature\ncurl -X POST localhost:9898/failpoints ... # 400: compiled without failpoints support\n\n# after\ncargo build -p neon_pageserver --features failpoints\ncurl -X POST localhost:9898/failpoints -d '[{\"name\":\"persistence-after-s3-sync\",\"actions\":\"return\"}]'","handlingStrategy":"validation","validationCode":"# Skip failpoint setup on builds without support:\nif curl -sf -X POST localhost:9898/failpoints -H 'Content-Type: application/json' \\\n -d '[]' >/dev/null 2>&1; then\n apply_failpoints\nelse\n echo \"failpoints not compiled in; rebuild with --features failpoints\"\nfi","typeGuard":null,"tryCatchPattern":"# Treat 400 'compiled without failpoints support' as a skip, not a failure:\nstatus=$(curl -s -o resp.txt -w '%{http_code}' -X POST localhost:9898/failpoints -d @fp.json)\nif [ \"$status\" = 400 ] && grep -q 'without failpoints support' resp.txt; then\n exit 0 # capability missing; not a test failure\nfi","preventionTips":["Run chaos/failpoint tests only against builds with --features failpoints","Mark failpoint scenarios skip-if-unavailable in CI","Keep the failpoints feature in test build profiles and out of release profiles deliberately"],"tags":["neon","http-utils","failpoints","build-features","testing"],"backgroundTag":"feature-not-compiled-in","analyzedSha":"8f60b04da47ffefe0e52bda2440134b42874eb75","analyzedAt":"2026-08-16T23:39:28.135Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}