dgraph-io/dgraph · error

Invalid tokenizer %s

Error message

Invalid tokenizer %s

What it means

Each tokenizer listed for an indexed predicate must be a registered tokenizer known to Dgraph (tok.GetTokenizer). If the name string doesn't match any built-in or loaded tokenizer, the schema update fails with "Invalid tokenizer <name>".

Source

Thrown at schema/parse.go:444

		}

		if !HasTokenizerOrVectorIndexSpec(schema) &&
			schema.Directive == pb.SchemaUpdate_INDEX {
			return errors.Errorf(
				"Require type of tokenizer for pred: %s of type: %s for indexing.",
				schema.Predicate, typ.Name())
		} else if HasTokenizerOrVectorIndexSpec(schema) &&
			schema.Directive != pb.SchemaUpdate_INDEX {
			return errors.Errorf("Tokenizers present without indexing on attr %s",
				x.ParseAttr(schema.Predicate))
		}
		// check for valid tokenizer types and duplicates
		var seen = make(map[string]bool)
		var seenSortableTok bool
		for _, t := range schema.Tokenizer {
			tokenizer, has := tok.GetTokenizer(t)
			if !has {
				return errors.Errorf("Invalid tokenizer %s", t)
			}
			tokenizerType, ok := types.TypeForName(tokenizer.Type())
			x.AssertTrue(ok) // Type is validated during tokenizer loading.
			if tokenizerType != typ {
				return errors.Errorf("Tokenizer: %s isn't valid for predicate: %s of type: %s",
					tokenizer.Name(), x.ParseAttr(schema.Predicate), typ.Name())
			}
			if _, ok := seen[tokenizer.Name()]; !ok {
				seen[tokenizer.Name()] = true
			} else {
				return errors.Errorf("Duplicate tokenizers present for attr %s",
					x.ParseAttr(schema.Predicate))
			}
			if tokenizer.IsSortable() {
				if seenSortableTok {
					return errors.Errorf("More than one sortable index encountered for: %v",
						schema.Predicate)
				}

View on GitHub (pinned to 759e242be6)

Solutions

1. Use a valid tokenizer name — for strings: hash, exact, term, fulltext, trigram, wikidata (where available); int/float/datetime/bool take their default single tokenizer.
2. Run the schema alteration again with corrected names.
3. Check the Dgraph docs for your version's supported tokenizer list.
4. Note the type must also match (see the paired 'isn't valid for predicate' error).

Example fix

// before
name: string @index(full-text) .
// after
name: string @index(fulltext) .

const VALID = new Set(['hash','exact','term','fulltext','trigram','int','float','datetime','year','month','day','hour','geo','default']);
function validateTokenizers(line) {
  const m = line.match(/@index\s*\(([^)]*)\)/);
  if (!m) return;
  for (const t of m[1].split(',').map(s => s.trim())) {
    if (!VALID.has(t)) throw new Error(`invalid tokenizer: ${t}`);
  }
}

try {
  await dgraph.Alter(ctx, op);
} catch (e) {
  if (String(e).includes('Invalid tokenizer')) {
    // correct the tokenizer name against your Dgraph version's docs
  }
  throw e;
}

When it happens

Trigger: Schema alteration listing a nonexistent tokenizer name, e.g. `@index(hashx)`, `@index(string)`, or misspellings like `@index(full-text)`; empty tokenizer strings; names removed/renamed between Dgraph versions.

Common situations: Typos in hand-edited DQL schema; copying tokenizers from other databases (e.g. 'trigram', 'ngram'); upgrading/downgrading Dgraph where a tokenizer was added or renamed; case-sensitivity mistakes ('Hash' vs 'hash').

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

• Authentication and authorization failures — expired tokens, bad credentials, and missing scopes.

Related errors

• illegal rune found "%c", expecting {
• Malformed JSON
• Require type of tokenizer for pred: %s of type: %s for index
• Tokenizers present without indexing on attr %s
• Tokenizer: %s isn't valid for predicate: %s of type: %s

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/4904a9b420565d3f. Report an issue: GitHub.