t8y2/dbx · error

token contains trailing data

Error message

token contains trailing data

What it means

After parsing identifier, password, kind, and service byte arrays from the decoded delegation token, the driver requires the remaining buffer to be empty. Leftover bytes indicate the input is longer than a valid Hadoop token structure — extra data, wrong encoding, or concatenated tokens.

Source

Thrown at agents/drivers/hive-go/config.go:796

	identifier, err := readHadoopByteArray(reader)
	if err != nil {
		return nil, nil, fmt.Errorf("identifier: %w", err)
	}
	password, err := readHadoopByteArray(reader)
	if err != nil {
		return nil, nil, fmt.Errorf("password: %w", err)
	}
	if len(identifier) == 0 || len(password) == 0 {
		return nil, nil, errors.New("token identifier and password must be non-empty")
	}
	if _, err := readHadoopByteArray(reader); err != nil {
		return nil, nil, fmt.Errorf("kind: %w", err)
	}
	if _, err := readHadoopByteArray(reader); err != nil {
		return nil, nil, fmt.Errorf("service: %w", err)
	}
	if reader.Len() != 0 {
		return nil, nil, errors.New("token contains trailing data")
	}
	return identifier, password, nil
}

func readHadoopByteArray(reader io.ByteReader) ([]byte, error) {
	length, err := readHadoopVInt(reader)
	if err != nil {
		return nil, err
	}
	if length < 0 {
		return nil, fmt.Errorf("negative length %d", length)
	}
	if length > 64*1024*1024 {
		return nil, fmt.Errorf("length %d exceeds limit", length)
	}
	value := make([]byte, int(length))
	byteReader, ok := reader.(io.Reader)
	if !ok {

View on GitHub (pinned to c0390bff16)

Solutions

1. Use the raw base64 token string exactly as produced by the Hadoop service, without extra quoting or wrappers.
2. Trim surrounding quotes/JSON from the value before passing it.
3. Re-generate the token and avoid manual editing; if tokens are concatenated, pass only one.

Example fix

// before
params["delegationToken"] = "\"" + token + "extra data\"" // quoted/wrapped
// after
params["delegationToken"] = token // exact base64 from Hive/Hadoop

func cleanTokenString(v string) string {
    return strings.Trim(strings.TrimSpace(v), "\"'")
}

When it happens

Trigger: Passing a token with trailing junk (whitespace is stripped, but other bytes are not), concatenating two tokens, encoding the token with an extra wrapper (e.g. JSON with quotes base64-encoded), or a version mismatch producing extra fields.

Common situations: Copy-paste including trailing characters that survive base64 decoding, pipelines that wrap the token in quotes or JSON before base64, older/newer Hadoop writing extra metadata.

Related errors

• token identifier and password must be non-empty
• Hive delegation token authentication requires delegationToke
• Hive delegation token authentication requires delegationToke
• token is empty
• decode Hive delegation token: %w

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/30af96ff685a9989. Report an issue: GitHub.