databendlabs/databend · error

failed to decode wkb data

Error message

failed to decode wkb data

What it means

`cast_scalar_to_variant` converts a Geometry scalar to VARIANT by decoding EWKB with `Ewkb::to_json()`; the `expect("failed to decode wkb data")` panics when the WKB/EWKB payload cannot be decoded. The code assumes any value stored in a Geometry column is valid EWKB.

Solutions

1. Validate the offending bytes with a WKB parser (e.g. postgis `ST_GeomFromWKB`, or wkb crate) to pinpoint the malformation.
2. Re-ingest the geometry from source, converting properly (e.g. ST_AsBinary / EWKB encoding) before loading into the Geometry column.
3. Check for ingestion paths that bypass geometry validation and add validation at load time.
4. Patch locally: replace `expect` with error propagation so bad rows surface as query errors naming the corrupt value instead of a panic.

Example fix

// before
let geom = Ewkb(bytes).to_json().expect("failed to decode wkb data");
// after
let geom = Ewkb(bytes).to_json().map_err(|e| ErrorCode::Internal(format!("failed to decode wkb data: {e}")))?;

// validate EWKB before casting: header magic byte + minimal length
fn looks_like_ewkb(b: &[u8]) -> bool {
    b.len() >= 5 && (b[0] == 0x00 || b[0] == 0x01)
}
// pre-check: looks_like_ewkb(geom_bytes)

fn is_valid_ewkb(bytes: &[u8]) -> bool { bytes.len() >= 5 && matches!(bytes[0], 0x00 | 0x01) }

When it happens

Trigger: Calling `cast_scalar_to_variant` on a `ScalarRef::Geometry(bytes)` whose bytes are not valid EWKB (truncated header, bad geometry type code, corrupted SRID/coords), or a GEOGRAPHY value hitting the same `Ewkb(bytes.0).to_json().expect(...)` in the next branch.

Common situations: External tools wrote non-EWKB bytes into a geometry column; data imported from another system (e.g. raw WKT or MySQL binary geometry) without conversion; byte corruption in storage or during copy; version mismatch in geometry encoding.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• The header tree can only contain DataHeader
• {}
• Temp table id used up
• Invalid temp table desc
• expect tuple type

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/b7d0e9e10bf50df9. Report an issue: GitHub.

Appendix: source

Thrown at src/query/expression/src/types/variant.rs:398

                    let values = cast_scalars_to_variants(fields, tz, None);
                    OwnedJsonb::build_object(
                        values
                            .iter()
                            .enumerate()
                            .map(|(i, bytes)| (format!("{}", i + 1), RawJsonb::new(bytes))),
                    )
                    .expect("failed to build jsonb object from tuple")
                }
            };
            buf.extend_from_slice(owned_jsonb.as_ref());
            return;
        }
        ScalarRef::Variant(bytes) => {
            buf.extend_from_slice(bytes);
            return;
        }
        ScalarRef::Geometry(bytes) => {
            let geom = Ewkb(bytes).to_json().expect("failed to decode wkb data");
            jsonb::parse_owned_jsonb_with_buf(geom.as_bytes(), buf)
                .expect("failed to parse geojson to json value");
            return;
        }
        ScalarRef::Geography(bytes) => {
            // todo: Implement direct conversion, omitting intermediate processes
            let geom = Ewkb(bytes.0).to_json().expect("failed to decode wkb data");
            jsonb::parse_owned_jsonb_with_buf(geom.as_bytes(), buf)
                .expect("failed to parse geojson to json value");
            return;
        }
        ScalarRef::Vector(scalar) => with_vector_number_type!(|NUM_TYPE| match scalar {
            VectorScalarRef::NUM_TYPE(vals) => {
                let items = cast_scalars_to_variants(
                    vals.iter()
                        .map(|n| ScalarRef::Number(NumberScalar::NUM_TYPE(*n))),
                    tz,
                    None,

View on GitHub (pinned to 288d84d76e)