{"record":{"id":"7d46d24a2706edd0","repo":"apache/cassandra","slug":"rebuilding-index-summary-because-offset-value-d","errorCode":null,"errorMessage":"Rebuilding index summary because offset value (%d) at position: %d is Big Endian while Little Endian is expected","messagePattern":"Rebuilding index summary because offset value \\((.+?)\\) at position: (.+?) is Big Endian while Little Endian is expected","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/io/sstable/indexsummary/IndexSummary.java","lineNumber":469,"sourceCode":"                FBUtilities.copy(in, new MemoryOutputStream(offsets), offsets.size());\n                FBUtilities.copy(in, new MemoryOutputStream(entries), entries.size());\n            }\n            catch (IOException ioe)\n            {\n                offsets.free();\n                entries.free();\n                throw ioe;\n            }\n\n            // Before 5.0 offsets were written using Native Endian, now they are stored as Little Endian,\n            // so we apply a heuristic here to detect\n            // if the loading index summary was created on a Big Endian machine using Native Endian format\n            if (offsets.size() > 0)\n            {\n                int offset = offsets.getInt(0);\n                int offsetReversed = Integer.reverseBytes(offset);\n                if (offsetReversed > 0 && offset > offsetReversed || offset - offsets.size() < 0)\n                    throw new IOException(String.format(\"Rebuilding index summary because offset value (%d) at position: %d \" +\n                                                        \"is Big Endian while Little Endian is expected\", offset, 0));\n            }\n            // our on-disk representation treats the offsets and the summary data as one contiguous structure,\n            // in which the offsets are based from the start of the structure. i.e., if the offsets occupy\n            // X bytes, the value of the first offset will be X. In memory we split the two regions up, so that\n            // the summary values are indexed from zero, so we apply a correction to the offsets when de/serializing.\n            // In this case subtracting X from each of the offsets.\n            for (int i = 0 ; i < offsets.size() ; i += 4)\n                offsets.setInt(i, (int) (offsets.getInt(i) - offsets.size()));\n            return new IndexSummary(partitioner, offsets, offsetCount, entries, entries.size(), fullSamplingSummarySize, minIndexInterval, samplingLevel);\n        }\n\n        /**\n         * Deserializes the first and last key stored in the summary\n         * <p>\n         * Only for use by offline tools like SSTableMetadataViewer, otherwise SSTable.first/last should be used.\n         */\n        public Pair<DecoratedKey, DecoratedKey> deserializeFirstLastKey(DataInputStreamPlus in, IPartitioner partitioner) throws IOException","sourceCodeStart":451,"sourceCodeEnd":487,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/io/sstable/indexsummary/IndexSummary.java#L451-L487","documentation":"Thrown by IndexSummary.deserialize as a heuristic endianness check: the first offset in a natively-written (little-endian) summary is inspected, and if it looks byte-reversed (Big Endian) the summary is rejected so it will be rebuilt. The on-disk summary must use little-endian (native on LE machines) offsets.","triggerScenarios":"Loading an IndexSummary.db written on a Big Endian machine (or by a non-native-ordering writer) onto a little-endian host; first offset value fails the sanity check offsetReversed > 0 && offset > offsetReversed, or first offset negative.","commonSituations":"Copying sstable files between architectures (e.g. POWER/SPARC big-endian hosts to x86), corrupted summary bytes coincidentally passing the check incorrectly.","solutions":["Let Cassandra rebuild the summary (delete Summary.db for the sstable or run upgradesstables so the summary is regenerated on the local architecture)","Do not copy sstables across different-endian architectures; re-stream data via repair instead","Restore summaries only from backups taken on the same architecture"],"exampleFix":"// before: copying files from big-endian host\nscp old-powerpc:/data/*.db /var/lib/cassandra/data/ks/t/\n// after: rebuild summaries locally\nrm /var/lib/cassandra/data/ks/t/*-Summary.db; nodetool refresh ks t","handlingStrategy":"fallback","validationCode":"// skip regeneration: delete Summary.db when moving sstables across architectures\nif (archOfSource != archOfLocal) Files.deleteIfExists(descriptor.fileFor(Components.SUMMARY).toPath());","typeGuard":null,"tryCatchPattern":"try { summary.deserialize(in, partitioner, minInterval, maxInterval); } catch (IOException e) { regenerateSummaryOnLocalArch(descriptor); }","preventionTips":["Never copy raw sstables between big-endian and little-endian hosts; stream via repair","Delete Summary.db components when in doubt — they regenerate","Pin architecture when restoring backups"],"tags":["io","sstable","index-summary","endianness"],"backgroundTag":"file-read-failed","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}