apache/hadoop · error · RuntimeException

Seeking in unsorted TFile

Error message

Seeking in unsorted TFile

What it means

Thrown by TFile.Reader.getBlockContainsKey(RawComparable, boolean) when a key-based seek is attempted on an unsorted TFile. Binary search over the block index requires sorted keys and a stored comparator; unsorted files have neither, so seeking by key is impossible and rejected with RuntimeException. The method backs reader/scanner lookups such as lowerBound/upperBound-style navigation.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java:998

    }

    /**
     * if greater is true then returns the beginning location of the block
     * containing the key strictly greater than input key. if greater is false
     * then returns the beginning location of the block greater than equal to
     * the input key
     * 
     * @param key
     *          the input key
     * @param greater
     *          boolean flag
     * @return
     * @throws IOException
     */
    Location getBlockContainsKey(RawComparable key, boolean greater)
        throws IOException {
      if (!isSorted()) {
        throw new RuntimeException("Seeking in unsorted TFile");
      }
      checkTFileDataIndex();
      int blkIndex =
          (greater) ? tfileIndex.upperBound(key) : tfileIndex.lowerBound(key);
      if (blkIndex < 0) return end;
      return new Location(blkIndex, 0);
    }

    Location getLocationByRecordNum(long recNum) throws IOException {
      checkTFileDataIndex();
      return tfileIndex.getLocationByRecordNum(recNum);
    }

    long getRecordNumByLocation(Location location) throws IOException {
      checkTFileDataIndex();
      return tfileIndex.getRecordNumByLocation(location);      
    }
    

View on GitHub (pinned to 2add963021)

Solutions

1. Rewrite the TFile in sorted mode (pass a comparator name to TFile.Writer) so key-based seeking is possible
2. For unsorted files, scan sequentially with Scanner.advance() and filter keys in application code
3. Check reader.isSorted() before any key seek and choose the sequential path automatically

Example fix

// before
Scanner scanner = reader.createScanner();
scanner.seekTo(key, 0, key.length); // RuntimeException: unsorted TFile

// after
Scanner scanner = reader.createScanner();
if (reader.isSorted()) {
  scanner.seekTo(key, 0, key.length);
} else {
  while (scanner.advance()) { /* compare scanner.entry().getKey() yourself */ }
}

if (!reader.isSorted()) {
  // key seek impossible: scan sequentially instead
  Scanner s = reader.createScanner();
  while (s.advance()) { /* match keys yourself */ }
} else {
  Scanner s = reader.createScanner();
  s.seekTo(key, 0, key.length);
}

static boolean supportsKeySeek(TFile.Reader reader) {
  return reader.isSorted();
}

catch (RuntimeException e) {
  if ("Seeking in unsorted TFile".equals(e.getMessage())) {
    // switch this read path to a full sequential scan
  }
}

When it happens

Trigger: Calling scanner.seekTo(key, 0, key.length) or reader APIs that locate a key's block on a file written with a null comparator.

Common situations: Point lookups (get by key) against a file produced by an unsorted writer; switching ingestion from sorted to unsorted while read paths still do seeks; testing seek code against unsorted fixtures.

Related errors

• Entries are not comparable for unsorted TFiles
• Cannot compare keys for unsorted TFiles.
• Attempt to seek before the begin location.
• Attempt to seek after the end location.
• Cannot seek to a negative offset " + targetPos

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/062d8adf127a428a. Report an issue: GitHub.