{"record":{"id":"396dc92619e61d71","repo":"pocketbase/pocketbase","slug":"unsupported-token-type-q","errorCode":null,"errorMessage":"unsupported token type %q","messagePattern":"unsupported token type %q","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"tools/search/filter.go","lineNumber":319,"sourceCode":"\t\tplaceholder := \"t\" + security.PseudorandomString(8)\n\n\t\treturn &ResolverResult{\n\t\t\tIdentifier: \"{:\" + placeholder + \"}\",\n\t\t\tParams:     dbx.Params{placeholder: cast.ToFloat64(token.Literal)},\n\t\t}, nil\n\tcase fexpr.TokenFunction:\n\t\tfn, ok := TokenFunctions[token.Literal]\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"unknown function %q\", token.Literal)\n\t\t}\n\n\t\targs, _ := token.Meta.([]fexpr.Token)\n\t\treturn fn(func(argToken fexpr.Token) (*ResolverResult, error) {\n\t\t\treturn resolveToken(argToken, fieldResolver)\n\t\t}, args...)\n\t}\n\n\treturn nil, fmt.Errorf(\"unsupported token type %q\", token.Type)\n}\n\n// Resolves = and != expressions in an attempt to minimize the COALESCE\n// usage and to gracefully handle null vs empty string normalizations.\n//\n// The expression `a = \"\" OR a is null` tends to perform better than\n// `COALESCE(a, \"\") = \"\"` since the direct match can be accomplished\n// with a seek while the COALESCE will induce a table scan.\nfunc resolveEqualExpr(equal bool, left, right *ResolverResult) dbx.Expression {\n\tequalOp := \"=\"\n\tnullEqualOp := \"IS\"\n\tconcatOp := \"OR\"\n\tnullExpr := \"IS NULL\"\n\tif !equal {\n\t\t// always use `IS NOT` instead of `!=` because direct non-equal comparisons\n\t\t// to nullable column values that are actually NULL yields to NULL instead of TRUE, eg.:\n\t\t// `'example' != nullableColumn` -> NULL even if nullableColumn row value is NULL\n\t\tequalOp = \"IS NOT\"","sourceCodeStart":301,"sourceCodeEnd":337,"githubUrl":"https://github.com/pocketbase/pocketbase/blob/5d217ddb50cb144d80a5d0b0bdf11b52b2c3e457/tools/search/filter.go#L301-L337","documentation":"Returned by tools/search's resolveToken when the filter token's type is none of Text, Number, or Function — the only token types this resolver can turn into dbx parameters/identifiers. It formats token.Type with %q, so the message carries the numeric type constant. Like the unknown-operator guard, this indicates grammar/resolver skew rather than ordinary user input: fexpr produced a token type the search package cannot compile.","triggerScenarios":"Effectively unreachable with the bundled fexpr, whose tokenizer only emits Text/Number/Function token types. It would surface with a custom or newer fexpr emitting new token kinds (e.g. raw regex or date tokens), or if Meta carried an unexpected type into a function path.","commonSituations":"Forked fexpr grammars; go.mod replace directives on fexpr; experimental branches adding token types ahead of resolver support.","solutions":["Align the fexpr dependency version with the tools/search version (pin both; no replace directives).","If maintaining a fork that adds token types, extend resolveToken with a case for each new type.","Sanitize/limit user filter input to the documented syntax so unsupported constructs never reach the resolver."],"exampleFix":"# before (go.mod)\nreplace github.com/pocketbase/fexpr =​> github.com/myorg/fexpr v0.0.0-custom\n# after\n# drop the replace, run:\ngo mod tidy","handlingStrategy":"validation","validationCode":"// constrain user filter input to the documented grammar before parsing\nvar filterRe = regexp.MustCompile(`^[a-zA-Z0-9_.\\s'\"=\\!~<>(),%-]+$`)\nif !filterRe.MatchString(rawFilter) {\n    return errors.New(\"filter contains unsupported syntax\")\n}","typeGuard":null,"tryCatchPattern":"if err != nil && strings.Contains(err.Error(), \"unsupported token type\") {\n    // grammar/resolver version skew: verify fexpr dependency pinning; reject input\n}","preventionTips":["Pin fexpr version; no forks/replace directives.","Whitelist filter syntax at the API boundary.","Add parser fuzz/property tests after grammar changes."],"tags":["go","search","filter","token","compat"],"backgroundTag":null,"analyzedSha":"5d217ddb50cb144d80a5d0b0bdf11b52b2c3e457","analyzedAt":"2026-08-15T10:06:33.165Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}