{"record":{"id":"2b6a4903e465ee5b","repo":"astrid-runtime/astrid","slug":"fuse-callback-probe-failed-code-message","errorCode":null,"errorMessage":"FUSE callback probe failed [{code}]: {message}","messagePattern":"FUSE callback probe failed \\[(.+?)\\]: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/astrid-storage-provider-fuse/src/service.rs","lineNumber":321,"sourceCode":"        request_id: format!(\"fuse-service-{}\", Uuid::new_v4()),\n        lease_token: launch.lease.lease_token.clone(),\n        operation: StorageFilesystemOperationV2::Stat {\n            path: String::new(),\n        },\n    };\n    let bytes = serde_json::to_vec(&request).context(\"encode FUSE callback probe\")?;\n    let length = u32::try_from(bytes.len()).context(\"FUSE callback probe is too large\")?;\n    stream.write_all(&length.to_be_bytes()).await?;\n    stream.write_all(&bytes).await?;\n    stream.flush().await?;\n    let response = read_callback_response(&mut stream).await?;\n    if response.request_id != request.request_id {\n        bail!(\"FUSE callback probe correlation mismatch\");\n    }\n    match response.outcome {\n        StorageFilesystemOutcomeV2::Success(_) => Ok(()),\n        StorageFilesystemOutcomeV2::Failure(StorageFilesystemFailureV1 { code, message }) => {\n            bail!(\"FUSE callback probe failed [{code}]: {message}\")\n        },\n    }\n}\n\nasync fn read_callback_response(stream: &mut LocalStream) -> Result<StorageFilesystemResponseV2> {\n    let mut length = [0_u8; 4];\n    stream.read_exact(&mut length).await?;\n    let length = u32::from_be_bytes(length) as usize;\n    if length == 0 || length > MAX_CALLBACK_BYTES {\n        bail!(\"FUSE callback response exceeds the bounded frame size\");\n    }\n    let mut bytes = vec![0_u8; length];\n    stream.read_exact(&mut bytes).await?;\n    let response: StorageFilesystemResponseV2 =\n        serde_json::from_slice(&bytes).context(\"decode FUSE callback probe\")?;\n    if response.protocol_version != STORAGE_FILESYSTEM_PROTOCOL_V2 {\n        bail!(\"FUSE callback probe protocol mismatch\");\n    }","sourceCodeStart":303,"sourceCodeEnd":339,"githubUrl":"https://github.com/astrid-runtime/astrid/blob/affd8760f44190dbdfbec23403f4c4b642c33112/crates/astrid-storage-provider-fuse/src/service.rs#L303-L339","documentation":"The FUSE filesystem callback probe round-trip completed, but the kernel-side callback reported a failure outcome instead of success. probe_callback() correlates the response by request_id and then bails with the failure's code and message, so the underlying remote failure text is surfaced here.","triggerScenarios":"Calling probe_callback() when the StorageFilesystemResponseV2 carries StorageFilesystemOutcomeV2::Failure(StorageFilesystemFailureV1 { code, message }) — i.e. the FUSE callback handler on the kernel/admin side rejected or failed the probe operation.","commonSituations":"The admin/kernel service is unhealthy or misconfigured when the provider starts and self-tests its callback channel; the callback endpoint lacks permissions or the view/mount referenced by the probe does not exist; a version mismatch causes the remote handler to fail the operation.","solutions":["Read the [{code}] and {message} embedded in the error — they come from the remote StorageFilesystemFailureV1 and identify the actual root cause.","Verify the admin/kernel service is running and healthy before launching the FUSE provider.","Check that the filesystem view/lease the probe refers to exists and the provider has access to it.","Re-run the launch (run_launch) after fixing the remote condition; the probe is a startup self-test and safe to retry."],"exampleFix":"// before: startup aborts on any probe failure\nprobe_callback(&mut stream, request).await?;\n// after: inspect the code and retry transient remote failures\nmatch probe_callback(&mut stream, request).await {\n    Ok(()) => {},\n    Err(e) if e.to_string().contains(\"[temporary]\") => {\n        tokio::time::sleep(Duration::from_secs(1)).await;\n        probe_callback(&mut stream, request).await?;\n    },\n    Err(e) => return Err(e),\n}","handlingStrategy":"try-catch","validationCode":"// No pre-call check available; the failure comes from the remote callback handler.\n// Optionally probe service health first if an admin status endpoint exists.","typeGuard":null,"tryCatchPattern":"// Rust\nmatch run_launch().await {\n    Err(e) if e.to_string().starts_with(\"FUSE callback probe failed\") => {\n        // parse \"[{code}]\" and handle known transient codes with retry\n    }\n    other => other?,\n}","preventionTips":["Ensure the admin/kernel service is healthy before launching the provider","Match provider and kernel versions to avoid handler-side failures","Treat the probe as a retryable startup self-test"],"tags":["fuse","callback","remote-failure","startup-probe"],"backgroundTag":"api-error-response","analyzedSha":"affd8760f44190dbdfbec23403f4c4b642c33112","analyzedAt":"2026-09-09T21:28:12.402Z","contentChangedAt":"2026-09-09T21:28:12.402Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}