{"record":{"id":"f71fa218c5c81ada","repo":"t8y2/dbx","slug":"token-contains-trailing-data","errorCode":null,"errorMessage":"token contains trailing data","messagePattern":"token contains trailing data","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"agents/drivers/argo-go/config.go","lineNumber":793,"sourceCode":"\tidentifier, err := readHadoopByteArray(reader)\n\tif err != nil {\n\t\treturn nil, nil, fmt.Errorf(\"identifier: %w\", err)\n\t}\n\tpassword, err := readHadoopByteArray(reader)\n\tif err != nil {\n\t\treturn nil, nil, fmt.Errorf(\"password: %w\", err)\n\t}\n\tif len(identifier) == 0 || len(password) == 0 {\n\t\treturn nil, nil, errors.New(\"token identifier and password must be non-empty\")\n\t}\n\tif _, err := readHadoopByteArray(reader); err != nil {\n\t\treturn nil, nil, fmt.Errorf(\"kind: %w\", err)\n\t}\n\tif _, err := readHadoopByteArray(reader); err != nil {\n\t\treturn nil, nil, fmt.Errorf(\"service: %w\", err)\n\t}\n\tif reader.Len() != 0 {\n\t\treturn nil, nil, errors.New(\"token contains trailing data\")\n\t}\n\treturn identifier, password, nil\n}\n\nfunc readHadoopByteArray(reader io.ByteReader) ([]byte, error) {\n\tlength, err := readHadoopVInt(reader)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif length < 0 {\n\t\treturn nil, fmt.Errorf(\"negative length %d\", length)\n\t}\n\tif length > 64*1024*1024 {\n\t\treturn nil, fmt.Errorf(\"length %d exceeds limit\", length)\n\t}\n\tvalue := make([]byte, int(length))\n\tbyteReader, ok := reader.(io.Reader)\n\tif !ok {","sourceCodeStart":775,"sourceCodeEnd":811,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/argo-go/config.go#L775-L811","documentation":"This library decodes Hive/Windows-style Hadoop delegation tokens, which are binary blobs (base64 in config) containing length-prefixed fields: identifier, password, kind, and service. After consuming all expected fields, any unread bytes left in the token buffer trigger this error, because the token does not match the expected Hadoop TokenIdentifier serialization and would silently produce corrupted credentials. It is thrown from decodeHadoopDelegationToken when the final reader.Len() != 0 check fails.","triggerScenarios":"Setting a delegation token (via delegationToken/token/password config passed to the Hive delegation token path) whose decoded bytes contain extra data after the identifier/password/kind/service fields — e.g. a token from a different Hadoop version, a wrapping format like a container envelope, or a copy/paste that included extra encoded bytes.","commonSituations":"Copying a token from a Hive/Hadoop job XML or debug output that includes surrounding fields; using a WebHCat or Oozie token that embeds the raw token in another structure; mixing tokens from incompatible Hadoop distributions; whitespace/newline handling removed but padding artifacts retained.","solutions":["Re-obtain the raw delegation token from the source (e.g. kinit/hs2 doAs token endpoint) and pass only the token itself, not an enclosing JSON/XML value.","Verify the token is the Hadoop TokenIdentifier bytes (identifier+password+kind+service), not the full Thrift/protobuf token container.","Check for accidental characters added during copy/paste or base64 re-encoding of an already-base64 value.","If the token comes from a different Hadoop major version, regenerate a token compatible with this driver."],"exampleFix":"// before\nconfig.DelegationToken = extractFromJSON(resp).entireBody\n// after\nconfig.DelegationToken = extractFromJSON(resp).delegationToken // only the token string","handlingStrategy":"validation","validationCode":"import (\"encoding/base64\"; \"strings\")\nfunc looksLikeRawHadoopToken(token string) error {\n\tb, err := base64.StdEncoding.DecodeString(strings.Join(strings.Fields(token), \"\"))\n\tif err != nil { return err }\n\tif len(b) < 8 { return errors.New(\"token suspiciously short\") }\n\treturn nil // pass only this token string to the driver\n}","typeGuard":null,"tryCatchPattern":"if err := driver.Connect(cfg); err != nil {\n\tif strings.Contains(err.Error(), \"trailing data\") {\n\t\t// re-fetch a fresh delegation token and retry once\n\t}\n\treturn err\n}","preventionTips":["Pass the token string exactly as issued by Hive/HS2 — never embed it in JSON/XML and pass the wrapper.","Do not re-base64 an already-base64 token.","Fetch a fresh token per deployment rather than copying from logs.","Confirm the token comes from the same Hadoop/Hive version family as the server."],"tags":["kerberos","delegation-token","serialization","config"],"backgroundTag":"delegation-token-malformed","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}