{"record":{"id":"97b812b93f919ff3","repo":"dgraph-io/dgraph","slug":"malformed-json","errorCode":null,"errorMessage":"Malformed JSON","messagePattern":"Malformed JSON","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"chunker/chunk.go","lineNumber":247,"sourceCode":"\t\t}\n\t}\n\tif _, err := out.WriteRune(']'); err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// consumeMap consumes the next map from the reader, and stores the result into the buffer out.\n// After ignoring spaces, if the reader does not begin with {, no rune will be consumed\n// from the reader.\nfunc (jc *jsonChunker) consumeMap(r *bufio.Reader, out *bytes.Buffer) error {\n\t// Just find the matching closing brace. Let the JSON-to-nquad parser in the mapper worry\n\t// about whether everything in between is valid JSON or not.\n\tdepth := 0\n\tfor {\n\t\tch, err := jc.nextRune(r)\n\t\tif err != nil {\n\t\t\treturn errors.New(\"Malformed JSON\")\n\t\t}\n\t\tif depth == 0 && ch != '{' {\n\t\t\t// We encountered a beginning rune that's not {,\n\t\t\t// unread the char and return without consuming anything.\n\t\t\tif err := r.UnreadRune(); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\n\t\tif _, err := out.WriteRune(ch); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tswitch ch {\n\t\tcase '{':\n\t\t\tdepth++\n\t\tcase '}':\n\t\t\tdepth--","sourceCodeStart":229,"sourceCodeEnd":265,"githubUrl":"https://github.com/dgraph-io/dgraph/blob/759e242be62c91f8d084da06ad0c8d21256d9c07/chunker/chunk.go#L229-L265","documentation":"GetTokenizers resolves each name in the given list via GetTokenizer and fails fast on the first unknown name. It is used e.g. by rebuildTokIndex to turn persisted tokenizer names back into Tokenizer instances. An unknown name means the schema references a tokenizer that does not exist in this build.","triggerScenarios":"Calling GetTokenizers with any string in `names` that is not one of the registered tokenizer names (\"exact\", \"hash\", \"term\", \"trigram\", \"fulltext\", etc.).","commonSituations":"Typo like \"fulltext \" with a space or wrong case in schema; schema created with a tokenizer removed in a newer Dgraph; manual schema edits during index rebuild.","solutions":["Fix the tokenizer name in the schema to a valid one (exact, hash, term, trigram, fulltext...).","Validate all tokenizer names with tok.GetTokenizer(name) before calling GetTokenizers.","Rebuild the index after correcting the schema so persisted names match the binary."],"exampleFix":"// before\ntoks, err := tok.GetTokenizers([]string{\"exact\", \"fulltx\"})\n// after\ntoks, err := tok.GetTokenizers([]string{\"exact\", \"fulltext\"})","handlingStrategy":"validation","validationCode":"for _, name := range names {\n    if _, found := tok.GetTokenizer(name); !found {\n        return fmt.Errorf(\"schema uses unknown tokenizer %q\", name)\n    }\n}\ntoks, err := tok.GetTokenizers(names)","typeGuard":null,"tryCatchPattern":"toks, err := tok.GetTokenizers(names)\nif err != nil {\n    var bad string\n    fmt.Sscanf(err.Error(), \"Invalid tokenizer %s\", &bad)\n    return fmt.Errorf(\"fix tokenizer %q in schema; valid: exact, hash, term, trigram, fulltext\", bad)\n}","preventionTips":["Validate all tokenizer names in the schema against GetTokenizer before rebuilding indices.","Trim/lowercase tokenizer names read from config or user input.","Regenerate persisted tokenizer lists after upgrading Dgraph."],"tags":["dgraph","tokenizer","index-rebuild","schema"],"backgroundTag":"invalid-tokenizer-name","analyzedSha":"759e242be62c91f8d084da06ad0c8d21256d9c07","analyzedAt":"2026-09-01T14:42:12.034Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}