apache/hadoop · error · IOException

Corrupted data index: negative block count

Error message

Corrupted data index: negative block count 

What it means

Error "Corrupted data index: negative block count " thrown in apache/hadoop.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/BCFile.java:876

   */
  static class DataIndex {
    final static String BLOCK_NAME = "BCFile.index";

    private final Algorithm defaultCompressionAlgorithm;

    // for data blocks, each entry specifies a block's offset, compressed size
    // and raw size
    private final ArrayList<BlockRegion> listRegions;

    // for read, deserialized from a file
    public DataIndex(DataInput in) throws IOException {
      defaultCompressionAlgorithm =
          Compression.getCompressionAlgorithmByName(
              Utils.readString(in, MAX_META_STRING_LENGTH));

      int n = Utils.readVInt(in);
      if (n < 0) {
        throw new IOException("Corrupted data index: negative block count "
            + n);
      }
      // Only a capacity hint: bound it so a corrupt count cannot force a huge
      // pre-allocation. The loop is limited by the bytes actually available.
      listRegions = new ArrayList<>(Math.min(n, 1024));

      for (int i = 0; i < n; i++) {
        BlockRegion region = new BlockRegion(in);
        listRegions.add(region);
      }
    }

    // for write
    public DataIndex(String defaultCompressionAlgorithmName) {
      this.defaultCompressionAlgorithm =
          Compression
              .getCompressionAlgorithmByName(defaultCompressionAlgorithmName);
      listRegions = new ArrayList<>();

View on GitHub (pinned to 2add963021)

Solutions

1. The data index is corrupted; regenerate the file and check for disk or transfer errors.

When it happens

Trigger: Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/BCFile.java:876 when the library encounters an invalid state.

Common situations: Occurs when the BCFile data index reports a negative block count, indicating corruption. Verify file integrity and treat as unrecoverable corrupt input.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ce1f349ec4bd62ce. Report an issue: GitHub.