databendlabs/databend · error · InvalidInput

{e}

Error message

{e}

What it means

IcebergFileIO::build_operator parses the table location string with the url crate before deriving the bucket and path. If the location is not a valid URI, the parse error's Display text is wrapped in an InvalidInput error and returned verbatim. This guards every downstream scheme/bucket extraction which assumes a parsed URL.

Solutions

  1. Read the wrapped message (it is the url::ParseError text, e.g. 'relative URL without a base') and fix the location string accordingly.
  2. Ensure the location includes scheme, host/bucket and path, e.g. 's3://my-bucket/warehouse/db/table'.
  3. Trim whitespace and validate the location before passing it to the registry.
  4. If the location comes from a catalog, re-register the table with a correct metadata location.

Example fix

// before
let (op, pos) = file_io.get_operator_path("s3:/my-bucket/table")?; // invalid: single slash, no host
// after
let (op, pos) = file_io.get_operator_path("s3://my-bucket/table")?;
Defensive patterns

Strategy: validation

Validate before calling

fn valid_location(loc: &str) -> bool {
    loc.trim() == loc && url::Url::parse(loc)
        .map(|u| !u.scheme().is_empty())
        .unwrap_or(false)
}

Try / catch

let location = location.trim();
let parsed = url::Url::parse(location)
    .map_err(|e| anyhow!("invalid iceberg location {location:?}: {e}"))?;

Prevention

When it happens

Trigger: Calling get_operator_path (or the iceberg_file_io_* test paths) with a location string that fails url::Url::parse, e.g. 's3://', 'bucket/path' without a scheme, spaces or illegal characters in the URI, or a malformed authority.

Common situations: Catalog metadata holding a truncated or hand-edited table location; a path stored without its scheme prefix; whitespace/newline contamination from config files or environment variables; non-UTF8-safe or percent-encoding issues in object keys.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/f829814d6245516a. Report an issue: GitHub.

Appendix: source

Thrown at src/common/storage/src/operator.rs:765

            return Err(Error::new(
                ErrorKind::InvalidInput,
                "s3.access-key-id and s3.secret-access-key must be configured together",
            ));
        }

        if has_session_token && !(has_access_key && has_secret_key) {
            return Err(Error::new(
                ErrorKind::InvalidInput,
                "s3.session-token requires s3.access-key-id and s3.secret-access-key",
            ));
        }

        Ok(())
    }

    fn build_operator(&self, location: &str) -> Result<(Operator, usize)> {
        let url = url::Url::parse(location)
            .map_err(|e| Error::new(ErrorKind::InvalidInput, e.to_string()))?;

        let scheme = url.scheme();

        // Handle file:// and memory:// URIs which don't have a host/bucket
        let is_local_scheme = matches!(scheme, "file" | "memory" | "");
        let (bucket, relative_path_pos) = if is_local_scheme {
            // For file:// URIs, the path starts after "file://"
            let prefix_len = if location.starts_with("file://") {
                7 // "file://".len()
            } else if location.starts_with("memory://") {
                9 // "memory://".len()
            } else {
                0
            };
            (None, prefix_len)
        } else {
            let bucket = url
                .host_str()

View on GitHub (pinned to 288d84d76e)