weaviate/weaviate · warning

asciiFoldIgnore requires asciiFold to be enabled

Error message

asciiFoldIgnore requires asciiFold to be enabled

What it means

validateAnalyzerConfig rejects a TextAnalyzerConfig where asciiFoldIgnore entries are supplied while asciiFold is disabled (false). The ignore list only has meaning as an exception list for ASCII folding, so it is invalid on its own and returns a 4xx error to the caller of genericTokenize.

Source

Thrown at adapters/handlers/rest/handlers_tokenize.go:251

		return schemaops.NewSchemaObjectsPropertiesTokenizeUnprocessableEntity().WithPayload(
			errPayloadFromSingleErr(principal, fmt.Errorf("unknown stopword preset %q; must be a built-in preset ('en', 'none') or defined in invertedIndexConfig.stopwordPresets", prop.TextAnalyzer.StopwordPreset)))
	}

	prepared := tokenizer.NewPreparedAnalyzer(prop.TextAnalyzer)
	result := tokenizer.Analyze(*params.Body.Text, prop.Tokenization, className, prepared, detector)

	return schemaops.NewSchemaObjectsPropertiesTokenizeOK().WithPayload(&models.TokenizeResponse{
		Indexed: result.Indexed,
		Query:   result.Query,
	})
}

func validateAnalyzerConfig(cfg *models.TextAnalyzerConfig) error {
	if cfg == nil {
		return nil
	}
	if !cfg.ASCIIFold && len(cfg.ASCIIFoldIgnore) > 0 {
		return fmt.Errorf("asciiFoldIgnore requires asciiFold to be enabled")
	}
	for _, entry := range cfg.ASCIIFoldIgnore {
		if utf8.RuneCountInString(norm.NFC.String(entry)) != 1 {
			return fmt.Errorf("each asciiFoldIgnore entry must be a single character, got %q", entry)
		}
	}
	return nil
}

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Set asciiFold: true in the same textAnalyzer config
2. Remove the asciiFoldIgnore array if folding is not wanted
3. Validate the analyzer config client-side before calling the endpoint

Example fix

// before
{"textAnalyzer":{"asciiFoldIgnore":["é"]}}
// after
{"textAnalyzer":{"asciiFold":true,"asciiFoldIgnore":["é"]}}

if (cfg?.asciiFoldIgnore?.length && !cfg.asciiFold) {
  throw new Error('asciiFoldIgnore requires asciiFold to be enabled')
}

function hasValidFoldConfig(cfg) {
  return !cfg?.asciiFoldIgnore?.length || cfg.asciiFold === true
}

try {
  return await api.tokenize(req)
} catch (e) {
  if (e.status === 422 && /asciiFoldIgnore requires asciiFold/.test(e.message)) {
    return api.tokenize({...req, textAnalyzer: {...req.textAnalyzer, asciiFold: true}})
  }
  throw e
}

When it happens

Trigger: POST tokenize (generic) with body.textAnalyzer containing a non-empty asciiFoldIgnore array but asciiFold absent or false.

Common situations: Developer adds ignore characters to preserve accented letters but forgets to also enable asciiFold; older clients that predate the asciiFold flag sending only the ignore list.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• unknown stopword preset %q; must be a built-in preset ('en',
• each asciiFoldIgnore entry must be a single character, got %
• invalid tokenization %q
• property %q is not a text type; filterable.tokenization only
• property %q already uses tokenization %q

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/ff282837792ecada. Report an issue: GitHub.