{"id":"19e62dfbb8e952b7","repo":"apache/kafka","slug":"encountered-corrupt-message-when-fetching-topic-pa","errorCode":null,"errorMessage":"Encountered corrupt message when fetching topic-partition ${tp.topicPartition()}","messagePattern":"Encountered corrupt message when fetching topic-partition (.+?)","errorType":"exception","errorClass":"KafkaException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareFetchCollector.java","lineNumber":182,"sourceCode":"            log.warn(\"Received unknown topic or partition error in fetch for partition {}.\", tp);\n            requestMetadataUpdate(metadata, subscriptions, tp.topicPartition());\n        } else if (error == Errors.UNKNOWN_TOPIC_ID) {\n            log.warn(\"Received unknown topic ID error in fetch for partition {}.\", tp);\n            requestMetadataUpdate(metadata, subscriptions, tp.topicPartition());\n        } else if (error == Errors.INCONSISTENT_TOPIC_ID) {\n            log.warn(\"Received inconsistent topic ID error in fetch for partition {}.\", tp);\n            requestMetadataUpdate(metadata, subscriptions, tp.topicPartition());\n        } else if (error == Errors.TOPIC_AUTHORIZATION_FAILED) {\n            // Log the actual partition and not just the topic to help with ACL propagation issues in large clusters\n            log.warn(\"Not authorized to read from partition {}.\", tp.topicPartition());\n            throw new TopicAuthorizationException(Set.of(tp.topic()));\n        } else if (error == Errors.UNKNOWN_LEADER_EPOCH) {\n            log.debug(\"Received unknown leader epoch error in fetch for partition {}.\", tp);\n        } else if (error == Errors.UNKNOWN_SERVER_ERROR) {\n            log.warn(\"Unknown server error while fetching topic-partition {}.\",\n                    tp.topicPartition());\n        } else if (error == Errors.CORRUPT_MESSAGE) {\n            throw new KafkaException(\"Encountered corrupt message when fetching topic-partition \"\n                    + tp.topicPartition());\n        } else {\n            throw new IllegalStateException(\"Unexpected error code \" + error.code()\n                    + \" while fetching from topic-partition \" + tp.topicPartition());\n        }\n    }\n}\n","sourceCodeStart":164,"sourceCodeEnd":190,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareFetchCollector.java#L164-L190","documentation":"Thrown by ShareFetchCollector.handleInitializeErrors when a fetch response returns Errors.CORRUPT_MESSAGE for a topic-partition. The broker reports that a fetched record could not be checksum-verified (CRC mismatch), indicating on-disk corruption or a transmission error; the share collector converts this into a KafkaException because individual record corruption is not retriable at the fetch layer without operator intervention.","triggerScenarios":"Disk corruption on a broker log segment; bit-flip during network transfer that survives into the broker; hardware fault producing a record whose CRC no longer matches; truncated/partial segment after a hard broker crash.","commonSituations":"Failing disk on a broker; memory or NIC errors causing silent corruption; running on storage without checksumming (e.g. some ephemeral cloud volumes); topic produced with acks=0 so producers never detected corruption before it landed.","solutions":["Investigate broker logs and disk health (dmesg, smartctl, broker log segments for the reported partition).","If corruption is isolated, use kafka-delete-records.sh or kafkacat to truncate past the bad offset, or restore the segment from a healthy replica.","Ensure replication factor >= 3 so a corrupt replica can be replaced from a healthy one.","Reproduce with the consumer's isolation.level and check producer client for retry/backoff configuration; verify the topic was produced with acks=all."],"exampleFix":"# before: corrupt offset in partition 'orders-3'\n\n# after: truncate past the corrupt offset\nbin/kafka-delete-records.sh --bootstrap-server broker:9092 \\\n  --offset-json-file delete-records.json\n# {\"partitions\":[{\"topic\":\"orders\",\"partition\":3,\"offset\":12345}],\"version\":1}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    records = consumer.poll(Duration.ofSeconds(5));\n} catch (org.apache.kafka.common.errors.KafkaException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"corrupt message\")) {\n        // A fetch returned a record that failed checksum / log validation on the broker side.\n        // This usually means disk corruption on a specific broker or a bug in a producer's serializer.\n        log.error(\"Corrupt message detected; will skip and continue\", e);\n        // Share consumers: RELEASE the affected partition's current batch on next poll so the\n        // broker can retire the records; quarantine the topic-partition for investigation.\n        quarantine(topicPartition);\n    } else {\n        throw e;\n    }\n}","preventionTips":["Investigate broker disk health and CRC errors in server.log when this fires.","Enable producer-side compression carefully; rare serializer bugs can produce unparseable payloads.","Do NOT retry the same fetch indefinitely; the corrupt record will keep failing.","Monitor this exception per topic-partition to localize the failing broker."],"tags":["share-consumer","data-corruption","broker","storage"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}