What causes "Bytes support in dynamic fields is not yet implemented" in quickwit-oss/tantivy?

Declare a bytes field in the schema and index the value there instead

Bytes support in dynamic fields is not yet implemented

Error message

Bytes support in dynamic fields is not yet implemented

What it means

index_json_value panics via unimplemented!() when a dynamically-typed JSON leaf is Bytes. Byte-array values cannot yet be indexed inside dynamic JSON fields; only schema-declared bytes fields support them. The panic is intentional placeholder behavior.

Source

Thrown at src/core/json_utils.rs:222

                postings_writer.subscribe(doc, 0u32, term_buffer, ctx);
            }
            ReferenceValueLeaf::Date(val) => {
                set_path_id(
                    term_buffer,
                    ctx.path_to_unordered_id
                        .get_or_allocate_unordered_id(json_path_writer.as_str()),
                );
                let val = val.truncate(DATE_TIME_PRECISION_INDEXED);
                term_buffer.append_type_and_fast_value(val);
                postings_writer.subscribe(doc, 0u32, term_buffer, ctx);
            }
            ReferenceValueLeaf::PreTokStr(_) => {
                unimplemented!(
                    "Pre-tokenized string support in dynamic fields is not yet implemented"
                )
            }
            ReferenceValueLeaf::Bytes(_) => {
                unimplemented!("Bytes support in dynamic fields is not yet implemented")
            }
            ReferenceValueLeaf::Facet(_) => {
                unimplemented!("Facet support in dynamic fields is not yet implemented")
            }
            ReferenceValueLeaf::IpAddr(_) => {
                unimplemented!("IP address support in dynamic fields is not yet implemented")
            }
            ReferenceValueLeaf::Custom(_) => {
                unimplemented!("the JSON field does not support custom field types")
            }
        },
        ReferenceValue::Array(elements) => {
            for val in elements {
                index_json_value(
                    doc,
                    val,
                    text_analyzer,
                    term_buffer,

View on GitHub (pinned to b5d8deb80c)

Solutions

Declare a bytes field in the schema and index the value there instead
Base64/hex-encode the bytes as a string inside the JSON field if approximate retrieval suffices
Drop the byte value from JSON documents before indexing

Example fix

// before
json_obj.insert("blob", Value::Bytes(bytes));
// after
let blob = schema_builder.add_bytes_field("blob", STORED); // typed field for bytes

Defensive patterns

Strategy: validation

Validate before calling

fn json_has_bytes(v: &serde_json::Value) -> bool {
    match v {
        serde_json::Value::Array(a) => a.iter().any(json_has_bytes),
        serde_json::Value::Object(o) => o.values().any(json_has_bytes),
        _ => false,
    }
}
// reject binary blobs before indexing: assert!(!json_has_bytes(&doc_value));

Type guard

fn is_bytes_leaf(v: &tantivy::schema::Value) -> bool {
    matches!(v.as_value(), tantivy::schema::ReferenceValueLeaf::Bytes(_))
}

Try / catch

let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| index_writer.add_document(doc)));
if result.is_err() { eprintln!("dynamic JSON field contained Bytes"); }

Prevention

Use schema-typed bytes fields for binary data
Store blobs in JSON only as base64/hex strings
Validate document payloads before add_document
Audit migrations that convert typed fields into JSON fields

When it happens

Trigger: Indexing a document whose JSON field value is a Bytes leaf (e.g. a tantivy Bytes value inserted into a JSON object) through index_document.

Common situations: Trying to store binary blobs inside JSON fields; migrating documents from typed fields to dynamic JSON fields without converting byte values.

Related errors

AI-assisted analysis of quickwit-oss/tantivy@b5d8deb80c (2026-09-05). Data as JSON: /api/errors/13eb8369ec552ac9. Report an issue: GitHub.