{"record":{"id":"87e632c56e780a13","repo":"xai-org/grok-build","slug":"upload-parked-credentials-rejected-http-401-re","errorCode":null,"errorMessage":"upload parked: credentials rejected (HTTP 401); retrying in background until auth recovers","messagePattern":"upload parked: credentials rejected \\(HTTP 401\\); retrying in background until auth recovers","errorType":"exception","errorClass":null,"httpStatus":401,"severity":"error","filePath":"crates/codegen/xai-file-utils/src/queue.rs","lineNumber":2135,"sourceCode":"                            tracing::warn!(\n                                attempt = item.attempts,\n                                parked,\n                                error = ?e,\n                                \"Auth error persists after credential refresh, aborting\"\n                            );\n                            return Err(e);\n                        };\n                        if !parked {\n                            parked = true;\n                            stats.auth_parked.fetch_add(1, Ordering::Relaxed);\n                            tracing::warn!(\n                                attempt = item.attempts,\n                                gcs_path = %item.gcs_path,\n                                \"401 persists after credential refresh; parking item until auth recovers\"\n                            );\n                            notify_completion(\n                                item,\n                                Err(anyhow::anyhow!(\n                                    \"upload parked: credentials rejected (HTTP 401); \\\n                                     retrying in background until auth recovers\"\n                                )),\n                            );\n                        }\n                        if item.enqueued_at.elapsed() >= policy.max_age {\n                            tracing::warn!(\n                                attempt = item.attempts,\n                                age_secs = item.enqueued_at.elapsed().as_secs(),\n                                \"Parked item exceeded max_age waiting for auth recovery, aborting\"\n                            );\n                            return Err(e);\n                        }\n                        wake = wait.await\n                            || (last_wire_attempt.elapsed() >= policy.auth_park_probe_interval\n                                && resolver.has_usable_credential());\n                    }\n                    if let Some(p) = permit.as_deref_mut() {","sourceCodeStart":2117,"sourceCodeEnd":2153,"githubUrl":"https://github.com/xai-org/grok-build/blob/bc7f02eddd3d84085849dc19ed216f11c23b0571/crates/codegen/xai-file-utils/src/queue.rs#L2117-L2153","documentation":"This error is delivered when the storage service keeps rejecting uploads with HTTP 401 even after the queue refreshes credentials. The worker parks the item and keeps retrying in the background until auth recovers, notifying the completion channel with this descriptive error so the caller knows the upload is deferred, not permanently failed. It is an authentication-state signal rather than a data or network fault.","triggerScenarios":"Calling upload/enqueue with credentials the GCS backend rejects (401) and the refresh path produces still-invalid credentials; expired service-account keys or revoked tokens persisting past refresh; clock skew making freshly issued tokens appear invalid.","commonSituations":"Expired or rotated service-account key files on disk; environment with stale cached tokens after a tenant/role change; system clock drift; IAM policy removing upload permission from the identity used.","solutions":["Verify and rotate the credentials file / token source the client uses; replace revoked service-account keys.","Check system clock sync (NTP) to avoid freshly issued tokens being rejected.","Confirm IAM permissions for uploads on the target bucket are intact for the authenticated identity.","Treat this completion error as 'parked, will retry' and monitor stats; re-check auth config while it retries in background."],"exampleFix":"// before\n// running with stale key\nexport GOOGLE_APPLICATION_CREDENTIALS=/path/to/old-key.json\n\n// after\nexport GOOGLE_APPLICATION_CREDENTIALS=/path/to/rotated-key.json\ngcloud auth activate-service-account --key-file=$GOOGLE_APPLICATION_CREDENTIALS","handlingStrategy":"try-catch","validationCode":"// Validate credentials are usable before enqueueing\nasync fn auth_ok(client: &StorageClient) -> bool {\n    client.check_auth().await.is_ok() // or a cheap authenticated probe\n}","typeGuard":null,"tryCatchPattern":"// Recognize the parked-upload error and defer\nif e.to_string().contains(\"credentials rejected (HTTP 401)\") {\n    tracing::warn!(\"upload parked pending auth recovery; will retry later\");\n    schedule_reauth_and_reenqueue();\n}","preventionTips":["Rotate service-account keys before expiry and reload them promptly.","Keep system clocks NTP-synced so fresh tokens validate.","Verify IAM upload permissions after role/policy changes.","Watch the 401 attribution hooks/fire_401_attribution signals to react early."],"tags":["rust","auth","http-401","upload-queue","credentials"],"backgroundTag":"auth-credentials-rejected-401","analyzedSha":"bc7f02eddd3d84085849dc19ed216f11c23b0571","analyzedAt":"2026-08-31T04:59:42.031Z","schemaVersion":2},"datasetVersion":"2026-08-31T09:17:48.483Z"}