apache/cassandra · error · MarshalException

Invalid Token:

Error message

Invalid Token: 

What it means

TokenUtf8Type validates that its UTF-8 string value can be converted back into a token by the current partitioner's token factory. If IPartitioner.getTokenFactory().fromString(str) returns null the value is rejected with this MarshalException.

Source

Thrown at src/java/org/apache/cassandra/db/marshal/TokenUtf8Type.java:42

import org.apache.cassandra.serializers.MarshalException;
import org.apache.cassandra.serializers.TypeSerializer;
import org.apache.cassandra.serializers.UTF8Serializer;
import org.apache.cassandra.utils.ByteBufferUtil;

import static org.apache.cassandra.config.DatabaseDescriptor.getPartitioner;

public class TokenUtf8Type extends PseudoUtf8Type
{
    public static final TokenUtf8Type instance = new TokenUtf8Type();
    static final TypeSerializer<String> tokenSerializer = new UTF8Serializer()
    {
        @Override
        public <V> void validate(V value, ValueAccessor<V> accessor) throws MarshalException
        {
            super.validate(value, accessor);
            String str = deserialize(value, accessor);
            if (null == getPartitioner().getTokenFactory().fromString(str))
                throw new MarshalException("Invalid Token: " + str);
        }
    };

    private static final ArgumentDeserializer ARGUMENT_DESERIALIZER = new DefaultArgumentDeserializer(instance);
    private static final ByteBuffer MASKED_VALUE = ByteBufferUtil.EMPTY_BYTE_BUFFER;

    TokenUtf8Type() {} // singleton

    String describe() { return "Token"; }

    @Override
    public TypeSerializer<String> getSerializer()
    {
        return tokenSerializer;
    }

    @Override
    public <VL, VR> int compareCustom(VL left, ValueAccessor<VL> accessorL, VR right, ValueAccessor<VR> accessorR)

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Generate token literals with the same partitioner's TokenFactory.toString/fromString rather than hand-computing.
2. Validate with getPartitioner().getTokenFactory().fromString(s) != null on the client before binding.
3. Trim/normalize whitespace and ensure the literal matches the partitioner's token format (e.g. signed 64-bit for Murmur3Partitioner).

Example fix

// before
String token = "-92233720368547758080"; // out of Murmur3 range
// after
Token t = Murmur3Partitioner.instance.getTokenFactory().fromString(token);
if (t == null) throw new IllegalArgumentException("invalid token: " + token);

boolean valid = tokenStr != null && DatabaseDescriptor.getPartitioner().getTokenFactory().fromString(tokenStr.trim()) != null;

try { type.validate(bb, ByteBufferAccessor.instance); } catch (MarshalException e) { /* invalid token literal */ }

When it happens

Trigger: Binding a value to a token-typed column (e.g. for token-range queries via the token pseudo-type as string) with a malformed token literal such as '-99999999999999999999999' or non-numeric text for RandomPartitioner/Murmur3 tokens.

Common situations: Hand-built token-range queries where the token string was computed with a different partitioner; copy/pasted tokens with whitespace or wrong digit counts; scripts generating out-of-range token literals.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

• Authentication and authorization failures — expired tokens, bad credentials, and missing scopes.

Related errors

• Token must be >= 0 and <= 2**127
• REVOKE operation is not supported by AllowAllAuthorizer
• %s is not a valid data resource name
• %s is not a valid function resource name
• Invalid value of auto_snapshot_ttl: ${conf.auto_snapshot_ttl

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/73a59afdf35d01ae. Report an issue: GitHub.