databendlabs/databend · error · InvalidInput

name_node is required for storage hdfs

Error message

name_node is required for storage hdfs

What it means

The hdfs:// scheme requires a name node address, but neither the URI (no host part) nor the connection options provided 'name_node'. parse_hdfs_params cannot construct a StorageHdfsConfig without it, so it fails with InvalidInput. The message notes the options-based form is preferred.

Solutions

1. Add the name_node connection option: CONNECTION = (name_node = 'name-node-host:8020').
2. Alternatively include the host in the URI: hdfs://name-node-host:8020/path/.
3. Verify the URI actually carries an authority (hdfs://host/path, not hdfs:///path) when relying on it.

Example fix

// before
CREATE STAGE s URL = 'hdfs:///warehouse/';
// after
CREATE STAGE s URL = 'hdfs:///warehouse/' CONNECTION = (name_node = 'nn.example.com:8020');

// Rust
let has_uri_nn = !uri_authority.is_empty();
let has_opt = conn_opts.contains_key("name_node");
if scheme == "hdfs" && !has_uri_nn && !has_opt { panic!("hdfs requires name_node"); }

When it happens

Trigger: A location like 'hdfs:///data/path' (empty authority) or 'hdfs://path' with no name_node option in CONNECTION, during CREATE STAGE or COPY binding.

Common situations: Copying S3-style stage definitions and switching the scheme to hdfs without adding name_node; forgetting that HDFS unlike object stores needs an explicit name node; URI with relative path but missing host.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• name_node in uri( ) and from connection option 'name_node'(…
• input uri is not a valid huggingface uri
• S3 params must remain S3
• The header tree can only contain DataHeader
• not implemented: Not implemented for storage type

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/0ec4ede49975fb29. Report an issue: GitHub.

Appendix: source

Thrown at src/query/sql/src/planner/binder/location.rs:389

    let name_node = match (name_node_option, name_node_from_uri) {
        (Some(n1), Some(n2)) => {
            if n1 != &n2 {
                return Err(Error::new(
                    ErrorKind::InvalidInput,
                    format!(
                        "name_node in uri({n2}) and from connection option 'name_node'({n1}) not match."
                    ),
                ));
            } else {
                n2
            }
        }
        (Some(n1), None) => n1.to_string(),
        (None, Some(n2)) => n2,
        (None, None) => {
            // we prefer user to specify name_node in options
            return Err(Error::new(
                ErrorKind::InvalidInput,
                "name_node is required for storage hdfs",
            ));
        }
    };
    let sp = StorageParams::Hdfs(databend_common_meta_app::storage::StorageHdfsConfig {
        name_node,
        root,
        network_config: None,
    });
    l.connection
        .check()
        .map_err(|err| Error::new(ErrorKind::InvalidInput, err.to_string()))?;
    Ok(sp)
}

// The FileSystem scheme of WebHDFS is “webhdfs://”. A WebHDFS FileSystem URI has the following format.
// webhdfs://<HOST>:<HTTP_PORT>/<PATH>

View on GitHub (pinned to 288d84d76e)