apache/cassandra · error · IllegalArgumentException

Unexpected string: ${tokenScanner.next()}

Error message

Unexpected string: ${tokenScanner.next()}

What it means

rangesForRebuildWithTokens parses the tokens string as a sequence of (start,end] range literals; after all matched ranges are consumed, any leftover input in the scanner means malformed trailing content, which is rejected.

Source

Thrown at src/java/org/apache/cassandra/service/Rebuild.java:206

    private static RangesAtEndpoint rangesForRebuildWithTokens(String tokens, String keyspace)
    {
        Token.TokenFactory factory = StorageService.instance.getTokenFactory();
        List<Range<Token>> ranges = new ArrayList<>();
        Pattern rangePattern = Pattern.compile("\\(\\s*(-?\\w+)\\s*,\\s*(-?\\w+)\\s*\\]");
        try (Scanner tokenScanner = new Scanner(tokens))
        {
            while (tokenScanner.findInLine(rangePattern) != null)
            {
                MatchResult range = tokenScanner.match();
                Token startToken = factory.fromString(range.group(1));
                Token endToken = factory.fromString(range.group(2));
                logger.info("adding range: ({},{}]", startToken, endToken);
                ranges.add(new Range<>(startToken, endToken));
            }
            if (tokenScanner.hasNext())
                throw new IllegalArgumentException("Unexpected string: " + tokenScanner.next());
        }

        // Ensure all specified ranges are actually ranges owned by this host
        RangesAtEndpoint localReplicas = StorageService.instance.getLocalReplicas(keyspace);
        RangesAtEndpoint.Builder streamRanges = new RangesAtEndpoint.Builder(getBroadcastAddressAndPort(), ranges.size());
        for (Range<Token> specifiedRange : ranges)
        {
            boolean foundParentRange = false;
            for (Replica localReplica : localReplicas)
            {
                if (localReplica.contains(specifiedRange))
                {
                    streamRanges.add(localReplica.decorateSubrange(specifiedRange));
                    foundParentRange = true;
                    break;
                }
            }
            if (!foundParentRange)

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Provide tokens strictly as comma-separated (start,end] ranges, e.g. "(-9223372036854775808,-4611686018427387904]","(-4611686018427387904,0]"
2. Check shell quoting so the whole tokens string reaches rebuild intact
3. Remove any trailing separators or non-range text

Example fix

// before
nodetool rebuild -ks ks1 --tokens "(-10,-5],"
// after (no trailing comma)
nodetool rebuild -ks ks1 --tokens "(-10,-5]"

// tokens must be only (start,end] ranges
Pattern RANGE = Pattern.compile("\\(([^,]+),([^]]+)\\]");
if (!RANGE.matcher(tokens).replaceAll("").trim().isEmpty())
    throw new IllegalArgumentException("malformed tokens argument: " + tokens);

try { rebuild(ks, tokens); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Unexpected string")) rebuild(ks, sanitizeTokens(tokens)); else throw e; }

When it happens

Trigger: Calling rebuild with a tokens argument containing anything after valid range tokens — e.g. trailing comma, stray text, or a bare token instead of a (start,end] pair.

Common situations: Hand-written token list with typos; quoting issues in shell causing partial parsing; mixing bare tokens with ranges; copy-paste introducing extra characters.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Commit log position must be given as <segment>,<position>
• Value '%s' can't be converted to integer.
• value %s is not a valid human-readable file size
• Cannot specify tokens without keyspace.
• This host was specified as a source for rebuilding. Sources

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/e2b2dd77095b190b. Report an issue: GitHub.