{"record":{"id":"2ac8bebfef7416b1","repo":"databendlabs/databend","slug":"storage-params-is-invalid-for-it-s-auto-detect-fai","errorCode":null,"errorMessage":"storage params is invalid for it's auto detect failed for {err:?}","messagePattern":"storage params is invalid for it's auto detect failed for (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/query/sql/src/planner/binder/location.rs","lineNumber":664,"sourceCode":"\n    // Validate every endpoint URL exposed by the resolved StorageParams\n    // (including secondary URLs such as OSS presign_endpoint_url) against the\n    // egress policy. Centralising the check here keeps each per-scheme parser\n    // free of policy knowledge and ensures new fields cannot silently bypass\n    // it.\n    //\n    // Run before sp.auto_detect(): the S3 auto-detect branch contacts the\n    // configured endpoint to discover the bucket region, so the policy must\n    // gate that call as well. auto_detect only mutates `region`, not the\n    // endpoint URL, so a single pre-check is sufficient.\n    check_storage_params_endpoints(&sp)\n        .await\n        .map_err(|err| Error::new(ErrorKind::InvalidInput, err.to_string()))?;\n\n    let sp = sp.auto_detect().await.map_err(|err| {\n        Error::new(\n            ErrorKind::InvalidInput,\n            anyhow!(\"storage params is invalid for it's auto detect failed for {err:?}\"),\n        )\n    })?;\n\n    Ok((sp, path))\n}\n\npub async fn get_storage_params_from_options(\n    options: &BTreeMap<String, String>,\n    resolved_connection: Option<UserDefinedConnection>,\n) -> databend_common_exception::Result<StorageParams> {\n    let location = options\n        .get(\"location\")\n        .ok_or_else(|| ErrorCode::BadArguments(\"missing option 'location'\".to_string()))?;\n    let connection = options.get(\"connection_name\");\n\n    let mut location = if let Some(connection_name) = connection {\n        let connection = resolved_connection.ok_or_else(|| {\n            ErrorCode::BadArguments(format!(","sourceCodeStart":646,"sourceCodeEnd":682,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/query/sql/src/planner/binder/location.rs#L646-L682","documentation":"After building `StorageParams` from the URI, the binder calls `.auto_detect()` which probes the storage backend (checks region, endpoint reachability, etc.). If that probing fails, the params are considered invalid and this error wraps the underlying cause. It usually means the configured endpoint/bucket is wrong or unreachable at plan time.","triggerScenarios":"Creating a stage or binding an external location where the resolved endpoint URL cannot be auto-detected: wrong region, non-existent bucket, unreachable custom endpoint, or missing credentials causing the probe to fail.","commonSituations":"Typo in bucket name, S3 bucket in a region different from the endpoint, on-prem MinIO/OSS endpoint behind a firewall or with a self-signed cert, DNS resolution failures inside the cluster.","solutions":["Inspect the wrapped `err:?` in the message for the concrete transport/region failure and fix that cause.","Explicitly specify ENDPOINT_URL and REGION in the stage/location options instead of relying on auto-detection.","Verify the bucket exists and is reachable from the query node (network, DNS, firewall, TLS).","Ensure credentials are provided (or the node's storage config supplies valid ones) so the probe can authenticate.","Test endpoint reachability with curl from the same node running the query."],"exampleFix":"// before\nURL = 's3://mybucket/data/'  -- relies on auto-detect, wrong region\n// after\nURL = 's3://mybucket/data/' ENDPOINT_URL = 'https://s3.us-east-1.amazonaws.com' REGION = 'us-east-1'","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"match parse_uri_location(&loc).await {\n    Err(e) if e.to_string().contains(\"auto detect failed\") => {\n        // inspect wrapped cause, check endpoint/bucket/network, then retry with explicit ENDPOINT_URL/REGION\n    }\n    Err(e) => return Err(e),\n    Ok(v) => v,\n}","preventionTips":["Always set ENDPOINT_URL and REGION explicitly for private/S3-compatible endpoints","Verify bucket names and network reachability from query nodes before creating stages","Keep credentials available in node config so auto-detect can authenticate"],"tags":["storage","network","auto-detect"],"backgroundTag":"http-request-failed","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}