What causes "Malformed JSON" in dgraph-io/dgraph?

Fix the tokenizer name in the schema to a valid one (exact, hash, term, trigram, fulltext...).

Malformed JSON

Error message

Malformed JSON

What it means

GetTokenizers resolves each name in the given list via GetTokenizer and fails fast on the first unknown name. It is used e.g. by rebuildTokIndex to turn persisted tokenizer names back into Tokenizer instances. An unknown name means the schema references a tokenizer that does not exist in this build.

Source

Thrown at chunker/chunk.go:247

		}
	}
	if _, err := out.WriteRune(']'); err != nil {
		return nil, err
	}
	return out, nil
}

// consumeMap consumes the next map from the reader, and stores the result into the buffer out.
// After ignoring spaces, if the reader does not begin with {, no rune will be consumed
// from the reader.
func (jc *jsonChunker) consumeMap(r *bufio.Reader, out *bytes.Buffer) error {
	// Just find the matching closing brace. Let the JSON-to-nquad parser in the mapper worry
	// about whether everything in between is valid JSON or not.
	depth := 0
	for {
		ch, err := jc.nextRune(r)
		if err != nil {
			return errors.New("Malformed JSON")
		}
		if depth == 0 && ch != '{' {
			// We encountered a beginning rune that's not {,
			// unread the char and return without consuming anything.
			if err := r.UnreadRune(); err != nil {
				return err
			}
			return nil
		}

		if _, err := out.WriteRune(ch); err != nil {
			return err
		}
		switch ch {
		case '{':
			depth++
		case '}':
			depth--

View on GitHub (pinned to 759e242be6)

Solutions

Fix the tokenizer name in the schema to a valid one (exact, hash, term, trigram, fulltext...).
Validate all tokenizer names with tok.GetTokenizer(name) before calling GetTokenizers.
Rebuild the index after correcting the schema so persisted names match the binary.

Example fix

// before
toks, err := tok.GetTokenizers([]string{"exact", "fulltx"})
// after
toks, err := tok.GetTokenizers([]string{"exact", "fulltext"})

Defensive patterns

Strategy: validation

Validate before calling

for _, name := range names {
    if _, found := tok.GetTokenizer(name); !found {
        return fmt.Errorf("schema uses unknown tokenizer %q", name)
    }
}
toks, err := tok.GetTokenizers(names)

Try / catch

toks, err := tok.GetTokenizers(names)
if err != nil {
    var bad string
    fmt.Sscanf(err.Error(), "Invalid tokenizer %s", &bad)
    return fmt.Errorf("fix tokenizer %q in schema; valid: exact, hash, term, trigram, fulltext", bad)
}

Prevention

Validate all tokenizer names in the schema against GetTokenizer before rebuilding indices.
Trim/lowercase tokenizer names read from config or user input.
Regenerate persisted tokenizer lists after upgrading Dgraph.

When it happens

Trigger: Calling GetTokenizers with any string in `names` that is not one of the registered tokenizer names ("exact", "hash", "term", "trigram", "fulltext", etc.).

Common situations: Typo like "fulltext " with a space or wrong case in schema; schema created with a tokenizer removed in a newer Dgraph; manual schema edits during index rebuild.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/97b812b93f919ff3. Report an issue: GitHub.