databendlabs/databend · error

Failed to create client

Error message

Failed to create client: {}

What it means

During verification, metaverifier spawns clients that connect to the meta service. If creating the meta gRPC client fails (network unreachable, wrong address, auth/TLS problems), the worker writes "ERROR" to the verifier result file, prints this message, and bails — failing the whole run.

Solutions

1. Confirm databend-meta is running and listening on the configured grpc_api_address (check logs, netstat/ss, curl).
2. Fix the address/port and TLS settings in the metaverifier config.
3. Check for connection-refused details in stderr before this message to identify the underlying cause.
4. Check the VERIFIER_RESULT_FILE for "ERROR" and rerun after connectivity is fixed.

Example fix

// before
grpc_api_address = "meta.internal:9191"   # host not resolvable
// after
grpc_api_address = "10.0.0.5:9191"        # reachable meta node

// probe the meta gRPC endpoint before spawning verifier clients
use tokio::net::TcpStream;
TcpStream::connect(&config.grpc_api_address)
    .await
    .map_err(|e| anyhow::anyhow!("meta gRPC address unreachable {}: {}", config.grpc_api_address, e))?;

let client = MetaGrpcClient::try_create(...)
    .map_err(|e| {
        fs::write(VERIFIER_RESULT_FILE, "ERROR").ok();
        eprintln!("Failed to create client: {} — check meta service is up on {}", e, addr);
        e
    })?;

When it happens

Trigger: grpc_api_address pointing at a host/port with no running databend-meta, DNS resolution failure, connection refused, TLS misconfiguration, or the meta service rejecting the client when creating a MetaGrpcClient inside the spawned task.

Common situations: Meta service not started or crashed; wrong port in grpc_api_address; firewall/Docker networking blocking the port; address configured with http:// prefix; version mismatch between metaverifier and the meta cluster.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• grpc_api_address MUST not be empty!
• verifier client error
• no metasrv running on
• no metasrv running on
• ContextVar has no value

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/402e3d4af0ea9f5f. Report an issue: GitHub.

Appendix: source

Thrown at src/meta/binaries/metaverifier/main.rs:146

        let handle = DatabendRuntime::spawn(
            async move {
                let client = MetaGrpcClient::<DatabendRuntime>::try_create(
                    addrs.clone(),
                    "root",
                    "xxx",
                    None,
                    None,
                    None,
                    DEFAULT_GRPC_MESSAGE_SIZE,
                );

                let client = match client {
                    Ok(client) => client,
                    Err(e) => {
                        fs::write(VERIFIER_RESULT_FILE, "ERROR")?;
                        eprintln!("Failed to create client: {}", e);
                        bail!("Failed to create client: {}", e);
                    }
                };

                verifier(
                    &client,
                    prefix,
                    config.number,
                    client_num,
                    config.remove_percent,
                )
                .await
            },
            None,
        );
        println!("verifier worker {} started..", client_num);
        handles.push(handle)
    }

View on GitHub (pinned to 288d84d76e)