{"record":{"id":"32b1fbb2435a0502","repo":"dgraph-io/dgraph","slug":"facet-s-w","errorCode":null,"errorMessage":"facet: %s: %w","messagePattern":"facet: (.+?): %w","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"chunker/json_parser.go","lineNumber":176,"sourceCode":"func parseScalarFacets(m map[string]interface{}, prefix string) ([]*api.Facet, error) {\n\t// This happens at root.\n\tif prefix == \"\" {\n\t\treturn nil, nil\n\t}\n\n\tvar facetsForPred []*api.Facet\n\tfor fname, facetVal := range m {\n\t\tif facetVal == nil {\n\t\t\tcontinue\n\t\t}\n\t\tif !strings.HasPrefix(fname, prefix) {\n\t\t\tcontinue\n\t\t}\n\n\t\tkey := fname[len(prefix):]\n\t\tfacet, err := handleBasicFacetsType(key, facetVal)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"facet: %s: %w\", fname, err)\n\t\t}\n\t\tfacetsForPred = append(facetsForPred, facet)\n\t}\n\n\treturn facetsForPred, nil\n}\n\n// This is the response for a map[string]interface{} i.e. a struct.\ntype mapResponse struct {\n\tuid       string       // uid retrieved or allocated for the node.\n\tnamespace uint64       // namespace to which the node belongs.\n\tfcts      []*api.Facet // facets on the edge connecting this node to the source if any.\n\trawFacets map[string]interface{}\n}\n\nfunc handleBasicType(k string, v interface{}, op int, nq *api.NQuad) error {\n\tswitch v := v.(type) {\n\tcase json.Number:","sourceCodeStart":158,"sourceCodeEnd":194,"githubUrl":"https://github.com/dgraph-io/dgraph/blob/759e242be62c91f8d084da06ad0c8d21256d9c07/chunker/json_parser.go#L158-L194","documentation":"After the arity check, GetTokens resolves the tokenizer by its byte ID via GetTokenizerByID. An unknown ID means the identifier byte is not one of the registered tokenizer identifiers (IdentExact, IdentHash, IdentTerm, IdentTrigram, IdentFullText, etc.), so tokenization cannot proceed.","triggerScenarios":"Calling tok.GetTokens(id, value) with a byte id that is not a registered tokenizer ID — e.g. an id read from corrupted index metadata or a hardcoded wrong constant.","commonSituations":"Index/posting metadata written by a different Dgraph version using different ID assignments; corrupted schema or index files; code inventing an ID instead of using tok package constants.","solutions":["Use the exported identifier constants (tok.IdentExact, tok.IdentTerm, ...) instead of literal bytes.","Rebuild the index if the stored tokenizer ID came from an incompatible older version.","Validate the id with tok.GetTokenizerByID(id) and handle !found gracefully before tokenizing."],"exampleFix":"// before\nconst id byte = 9\ntokens, err := tok.GetTokens(id, val)\n// after\nid := byte(tok.IdentExact)\ntokens, err := tok.GetTokens(id, val)","handlingStrategy":"validation","validationCode":"if _, ok := tok.GetTokenizerByID(id); !ok {\n    return fmt.Errorf(\"tokenizer id %d not registered\", id)\n}","typeGuard":null,"tryCatchPattern":"tokens, err := tok.GetTokens(id, value)\nif err != nil && strings.Contains(err.Error(), \"No tokenizer was found with id\") {\n    return nil, fmt.Errorf(\"tokenizer id %d unknown; rebuild index or use tok constants\", id)\n}","preventionTips":["Use exported constants (IdentExact, IdentTerm, ...) instead of literal byte IDs.","Rebuild indices after Dgraph version upgrades if IDs may differ.","Guard tokenizer-ID reads from persisted metadata with GetTokenizerByID."],"tags":["dgraph","tokenizer","identifier","version-mismatch"],"backgroundTag":"unknown-tokenizer-id","analyzedSha":"759e242be62c91f8d084da06ad0c8d21256d9c07","analyzedAt":"2026-09-01T14:42:12.034Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}