What causes "Failed to validate a hint for {}: {} - skipped" in apache/cassandra?

Run schema agreement (DESCRIBE SCHEMA diff) between source and target nodes and align schemas

Failed to validate a hint for {}: {} - skipped

Error message

Failed to validate a hint for {}: {} - skipped

What it means

When a hints message arrives, the handler validates that the hinted mutation's partition updates pass schema validation. If validation raises a MarshalException (e.g. the data no longer matches the local schema), the hint is considered invalid: it is logged with this warning, the sender is responded to as if delivered, and the hint is skipped rather than applied. This prevents corrupt or schema-incompatible hinted writes from failing the whole replay.

Source

Thrown at src/java/org/apache/cassandra/hints/HintVerbHandler.java:82

        if (hint == null)
        {
            if (logger.isTraceEnabled())
                logger.trace("Failed to decode and apply a hint for {}: {} - table with id {} is unknown",
                             address,
                             hostId,
                             message.payload.unknownTableID);
            respond(message);
            return;
        }

        // We must perform validation before applying the hint, and there is no other place to do it other than here.
        try
        {
            hint.mutation.getPartitionUpdates().forEach(PartitionUpdate::validate);
        }
        catch (MarshalException e)
        {
            logger.warn("Failed to validate a hint for {}: {} - skipped", address, hostId);
            respond(message);
            return;
        }

        ClusterMetadata metadata = ClusterMetadata.current();
        NodeId localId = metadata.myNodeId();
        if (!hostId.equals(localId.toUUID()) && !hostId.equals(metadata.directory.hostId(localId)))
        {
            // the hint may have been written prior to upgrading, in which case it would be addressing the old
            // host id for its target node. If the id in the hint matches neither the pre-upgrade host id nor the
            // post-upgrade node id for this peer, the node is not the final destination of the hint (must have gotten
            // it from a decommissioning node), so just store it locally, to be delivered later.
            HintsService.instance.write(hostId, hint);
            respond(message);
        }
        else if (!StorageProxy.instance.appliesLocally(hint.mutation))
        {
            // the topology has changed, and we are no longer a replica of the mutation - since we don't know which node(s)

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Run schema agreement (DESCRIBE SCHEMA diff) between source and target nodes and align schemas
Check which tableId/table the hint targets and whether the table was altered or dropped
If schema drift, run a repair/rebuild of the affected node instead of relying on hints
Verify all nodes run compatible Cassandra versions

Example fix

// before: altering a column type while hints pending
ALTER TABLE users ALTER age TYPE text;
// after: drop and repopulate via repair instead, or drain hints first
nodetool drain; -- then alter schema, then restart so hints are re-validated/created

Defensive patterns

Strategy: try-catch

Validate before calling

// ensure schema agreement before relying on hinted handoff
// on operator side:
//   nodetool describecluster  -> check schema versions agree

Try / catch

try {
  hint.mutation.getPartitionUpdates().forEach(PartitionUpdate::validate);
} catch (MarshalException e) {
  logger.warn("Skipping invalid hint: {}", e.getMessage());
  respond(message); // treat as delivered, do not retry
}

Prevention

Keep schema versions in agreement across the cluster before schema changes
Drain/flush hints before incompatible schema migrations
Run repairs after schema changes to backfill skipped hints
Keep node versions compatible

When it happens

Trigger: A hint's mutation fails PartitionUpdate.validate() due to MarshalException — schema drift between sender and receiver, corrupted hint payload, or a table altered/dropped between hint creation and delivery.

Common situations: Schema changes (column type altered, table dropped) on the target node while hints were queued; mixing nodes with incompatible schema versions; corrupted hint files replayed after a crash.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/a96581c543da6dd7. Report an issue: GitHub.