{"record":{"id":"9388702d813a6bd0","repo":"quickwit-oss/quickwit","slug":"tags-collection-is-only-allowed-on-text-fields-wit","errorCode":null,"errorMessage":"tags collection is only allowed on text fields with the `raw` tokenizer","messagePattern":"tags collection is only allowed on text fields with the `raw` tokenizer","errorType":"validation","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-doc-mapper/src/doc_mapper/doc_mapper_impl.rs","lineNumber":327,"sourceCode":"/// - the field must be indexed.\nfn validate_tag(tag_field_name: &str, schema: &Schema) -> Result<(), anyhow::Error> {\n    if tag_field_name.starts_with('.') || tag_field_name.starts_with(\"\\\\.\") {\n        bail!(\"tag field `{tag_field_name}` should not start with a `.`\");\n    }\n    if tag_field_name.ends_with('.') {\n        bail!(\"tag field `{tag_field_name}` should not end with a `.`\");\n    }\n    let field = schema\n        .get_field(tag_field_name)\n        .with_context(|| format!(\"unknown tag field: `{tag_field_name}`\"))?;\n    let field_type = schema.get_field_entry(field).field_type();\n    match field_type {\n        FieldType::Str(options) => {\n            let tokenizer_opt = options\n                .get_indexing_options()\n                .map(|text_options: &tantivy::schema::TextFieldIndexing| text_options.tokenizer());\n            if tokenizer_opt != Some(RAW_TOKENIZER_NAME) {\n                bail!(\"tags collection is only allowed on text fields with the `raw` tokenizer\");\n            }\n        }\n        FieldType::U64(_) | FieldType::I64(_) => {\n            // u64 and i64 are accepted as tags.\n        }\n        _ => {\n            // We avoid the bytes / bool / f64 types,\n            // as they are generally speaking poor tags and we want to avoid\n            // bugs associated to the multiplicity of their representation.\n            //\n            // (Tags are relying heavily on string manipulation and we want to\n            // avoid a \"ZRP because you searched you searched for 0.100 instead of 0.1\",\n            // or `myflag:1`, `myflag:True` instead of `myflag:true`.\n            bail!(\n                \"tags collection is not allowed on `{}` fields\",\n                field_type.value_type().name().to_lowercase()\n            )\n        }","sourceCodeStart":309,"sourceCodeEnd":345,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-doc-mapper/src/doc_mapper/doc_mapper_impl.rs#L309-L345","documentation":"For text (Str) fields, tags are only allowed when the field is indexed with the `raw` tokenizer. Tokenized text fields have multiple token representations of a value, so tag equality filtering would be unreliable; validate_tag enforces the raw tokenizer for Str fields.","triggerScenarios":"`QuickwitDocMapper::try_from` where a `tag_fields` entry is a Str field whose text indexing options use a tokenizer other than `raw` (e.g. the default tokenizer), or has no indexing options at all.","commonSituations":"Making a normal full-text field (e.g. `message`) a tag field, forgetting that tags require raw tokenization.","solutions":["Configure the field's text indexing with `tokenizer: raw`.","Or use a separate raw-tokenized field (or u64/i64 field) as the tag field."],"exampleFix":"# before\n- name: service\n  type: text\n  indexed: true\n\ntag_fields: [service]\n\n# after\n- name: service\n  type: text\n  fieldnorms: false\n  text_type:\n    index: true\n    tokenizer: raw\n\ntag_fields: [service]","handlingStrategy":"validation","validationCode":"fn check_text_tags_use_raw(cfg: &serde_yaml::Value) -> Result<(), String> {\n    let tags: Vec<&str> = cfg[\"doc_mapping\"][\"tag_fields\"].as_sequence()\n        .map(|s| s.iter().filter_map(|v| v.as_str()).collect()).unwrap_or_default();\n    if let Some(fields) = cfg[\"doc_mapping\"][\"field_mappings\"].as_sequence() {\n        for f in fields {\n            let name = f[\"name\"].as_str().unwrap_or_default();\n            if tags.contains(&name) && f[\"type\"].as_str() == Some(\"text\") {\n                let tok = f[\"text_type\"][\"tokenizer\"].as_str().unwrap_or(\"default\");\n                if tok != \"raw\" {\n                    return Err(format!(\"tag field `{name}` (text) must use the raw tokenizer\"));\n                }\n            }\n        }\n    }\n    Ok(())\n}","typeGuard":null,"tryCatchPattern":"if let Err(e) = QuickwitDocMapper::try_from(&index_config) {\n    if e.to_string().contains(\"raw tokenizer\") {\n        eprintln!(\"Set tokenizer: raw on tag text fields: {e}\");\n    }\n    return Err(e);\n}","preventionTips":["Only tag text fields configured with `tokenizer: raw`.","For full-text fields needing tags, create a dedicated raw companion field.","Prefer u64/i64 ids over text for tag fields when possible."],"tags":["config","tag-fields","tokenizer"],"backgroundTag":"invalid-config-value","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}