quickwit-oss/quickwit · error · anyhow::Error

concatenate field has `include_dynamic_fields` set, but inde

Error message

concatenate field has `include_dynamic_fields` set, but index isn't dynamic

What it means

During DocMapper construction (try_from), the mapping tree is built and may produce concatenate field mappings that include dynamic fields. If any such mapping exists but the index schema defines no dynamic field, construction is aborted. A concatenate field with `include_dynamic_fields: true` only makes sense when the index actually has a dynamic field to pull from.

Source

Thrown at quickwit/quickwit-doc-mapper/src/doc_mapper/doc_mapper_impl.rs:194

        let document_size_field = if doc_mapping.store_document_size {
            let document_size_field_options = tantivy::schema::NumericOptions::default().set_fast();
            Some(
                schema_builder.add_u64_field(DOCUMENT_SIZE_FIELD_NAME, document_size_field_options),
            )
        } else {
            None
        };
        let source_field = if doc_mapping.store_source {
            Some(schema_builder.add_json_field(SOURCE_FIELD_NAME, STORED))
        } else {
            None
        };
        let MappingNodeRoot {
            field_mappings,
            concatenate_dynamic_fields,
        } = build_mapping_tree(&doc_mapping.field_mappings, &mut schema_builder)?;
        if !concatenate_dynamic_fields.is_empty() && dynamic_field.is_none() {
            bail!("concatenate field has `include_dynamic_fields` set, but index isn't dynamic");
        }
        let timestamp_field_path = if let Some(timestamp_field_name) = &doc_mapping.timestamp_field
        {
            validate_timestamp_field(timestamp_field_name, &field_mappings)?;
            Some(build_field_path_from_str(timestamp_field_name))
        } else {
            None
        };
        let schema = schema_builder.build();

        let tokenizer_manager = create_default_quickwit_tokenizer_manager();
        let mut custom_tokenizer_names = HashSet::new();
        for tokenizer_config_entry in &doc_mapping.tokenizers {
            if custom_tokenizer_names.contains(&tokenizer_config_entry.name) {
                bail!(
                    "duplicated custom tokenizer: `{}`",
                    tokenizer_config_entry.name
                );

View on GitHub (pinned to a39730c5cd)

Solutions

Add a `dynamic_field` entry to the doc_mapping so the index is dynamic.
Set `include_dynamic_fields: false` (or remove it) on the concatenate field.
Remove the concatenate field mapping if it is no longer needed.

Example fix

# before
doc_mapping:
  mode: lenient
  concatenate:
    body_concat:
      fields: [body]
      include_dynamic_fields: true

# after
doc_mapping:
  mode: lenient
  dynamic_field: dynamic
  concatenate:
    body_concat:
      fields: [body]
      include_dynamic_fields: true

Defensive patterns

Strategy: validation

Validate before calling

fn check_concatenate_dynamic(cfg: &serde_yaml::Value) -> Result<(), String> {
    let has_dynamic = cfg["doc_mapping"]["dynamic_field"].is_some();
    let concat = &cfg["doc_mapping"]["concatenate"];
    if let Some(concat) = concat.as_mapping() {
        for (_name, entry) in concat {
            if entry["include_dynamic_fields"].as_bool() == Some(true) && !has_dynamic {
                return Err("concatenate uses include_dynamic_fields but no dynamic_field is set".into());
            }
        }
    }
    Ok(())
}

Try / catch

match QuickwitDocMapper::try_from(index_config) {
    Ok(mapper) => mapper,
    Err(e) if e.to_string().contains("include_dynamic_fields") => {
        eprintln!("Config error: add a dynamic_field or disable include_dynamic_fields: {e}");
        return Err(e);
    }
    Err(e) => return Err(e),
}

Prevention

Treat `include_dynamic_fields: true` as implying a dynamic_field must exist in the same doc_mapping.
Validate index configs with `quickwit index create --dry-run` style checks before deploy.
When removing a dynamic field, grep the config for `include_dynamic_fields`.

When it happens

Trigger: Calling `QuickwitDocMapper::try_from(index_config)` where the index config's doc_mapping contains a concatenate field entry with `include_dynamic_fields: true` while `doc_mapping.dynamic_field` is unset.

Common situations: Copy-pasting an index config that uses concatenate + dynamic fields from an example, then removing or renaming the dynamic field without removing `include_dynamic_fields: true` from the concatenate mapping.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/1fc6868a12e77e5e. Report an issue: GitHub.