t8y2/dbx · error

token contains trailing data

Error message

token contains trailing data

What it means

This library decodes Hive/Windows-style Hadoop delegation tokens, which are binary blobs (base64 in config) containing length-prefixed fields: identifier, password, kind, and service. After consuming all expected fields, any unread bytes left in the token buffer trigger this error, because the token does not match the expected Hadoop TokenIdentifier serialization and would silently produce corrupted credentials. It is thrown from decodeHadoopDelegationToken when the final reader.Len() != 0 check fails.

Source

Thrown at agents/drivers/argo-go/config.go:793

	identifier, err := readHadoopByteArray(reader)
	if err != nil {
		return nil, nil, fmt.Errorf("identifier: %w", err)
	}
	password, err := readHadoopByteArray(reader)
	if err != nil {
		return nil, nil, fmt.Errorf("password: %w", err)
	}
	if len(identifier) == 0 || len(password) == 0 {
		return nil, nil, errors.New("token identifier and password must be non-empty")
	}
	if _, err := readHadoopByteArray(reader); err != nil {
		return nil, nil, fmt.Errorf("kind: %w", err)
	}
	if _, err := readHadoopByteArray(reader); err != nil {
		return nil, nil, fmt.Errorf("service: %w", err)
	}
	if reader.Len() != 0 {
		return nil, nil, errors.New("token contains trailing data")
	}
	return identifier, password, nil
}

func readHadoopByteArray(reader io.ByteReader) ([]byte, error) {
	length, err := readHadoopVInt(reader)
	if err != nil {
		return nil, err
	}
	if length < 0 {
		return nil, fmt.Errorf("negative length %d", length)
	}
	if length > 64*1024*1024 {
		return nil, fmt.Errorf("length %d exceeds limit", length)
	}
	value := make([]byte, int(length))
	byteReader, ok := reader.(io.Reader)
	if !ok {

View on GitHub (pinned to c0390bff16)

Solutions

1. Re-obtain the raw delegation token from the source (e.g. kinit/hs2 doAs token endpoint) and pass only the token itself, not an enclosing JSON/XML value.
2. Verify the token is the Hadoop TokenIdentifier bytes (identifier+password+kind+service), not the full Thrift/protobuf token container.
3. Check for accidental characters added during copy/paste or base64 re-encoding of an already-base64 value.
4. If the token comes from a different Hadoop major version, regenerate a token compatible with this driver.

Example fix

// before
config.DelegationToken = extractFromJSON(resp).entireBody
// after
config.DelegationToken = extractFromJSON(resp).delegationToken // only the token string

import ("encoding/base64"; "strings")
func looksLikeRawHadoopToken(token string) error {
	b, err := base64.StdEncoding.DecodeString(strings.Join(strings.Fields(token), ""))
	if err != nil { return err }
	if len(b) < 8 { return errors.New("token suspiciously short") }
	return nil // pass only this token string to the driver
}

if err := driver.Connect(cfg); err != nil {
	if strings.Contains(err.Error(), "trailing data") {
		// re-fetch a fresh delegation token and retry once
	}
	return err
}

When it happens

Trigger: Setting a delegation token (via delegationToken/token/password config passed to the Hive delegation token path) whose decoded bytes contain extra data after the identifier/password/kind/service fields — e.g. a token from a different Hadoop version, a wrapping format like a container envelope, or a copy/paste that included extra encoded bytes.

Common situations: Copying a token from a Hive/Hadoop job XML or debug output that includes surrounding fields; using a WebHCat or Oozie token that embeds the raw token in another structure; mixing tokens from incompatible Hadoop distributions; whitespace/newline handling removed but padding artifacts retained.

Related errors

• Kerberos requires krb5.conf or Windows SSPI
• Kerberos requires SSPI, credential cache, keytab, or princip
• Kerberos JAAS config contains no Krb5LoginModule
• token identifier and password must be non-empty
• token contains trailing data

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/f71fa218c5c81ada. Report an issue: GitHub.