{"record":{"id":"1316985400e0c382","repo":"apache/hadoop","slug":"cannot-compare-keys-for-unsorted-tfiles","errorCode":null,"errorMessage":"Cannot compare keys for unsorted TFiles.","messagePattern":"Cannot compare keys for unsorted TFiles\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java","lineNumber":1019,"sourceCode":"      int blkIndex =\n          (greater) ? tfileIndex.upperBound(key) : tfileIndex.lowerBound(key);\n      if (blkIndex < 0) return end;\n      return new Location(blkIndex, 0);\n    }\n\n    Location getLocationByRecordNum(long recNum) throws IOException {\n      checkTFileDataIndex();\n      return tfileIndex.getLocationByRecordNum(recNum);\n    }\n\n    long getRecordNumByLocation(Location location) throws IOException {\n      checkTFileDataIndex();\n      return tfileIndex.getRecordNumByLocation(location);      \n    }\n    \n    int compareKeys(byte[] a, int o1, int l1, byte[] b, int o2, int l2) {\n      if (!isSorted()) {\n        throw new RuntimeException(\"Cannot compare keys for unsorted TFiles.\");\n      }\n      return comparator.compare(a, o1, l1, b, o2, l2);\n    }\n\n    int compareKeys(RawComparable a, RawComparable b) {\n      if (!isSorted()) {\n        throw new RuntimeException(\"Cannot compare keys for unsorted TFiles.\");\n      }\n      return comparator.compare(a, b);\n    }\n\n    /**\n     * Get the location pointing to the beginning of the first key-value pair in\n     * a compressed block whose byte offset in the TFile is greater than or\n     * equal to the specified offset.\n     * \n     * @param offset\n     *          the user supplied offset.","sourceCodeStart":1001,"sourceCodeEnd":1037,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java#L1001-L1037","documentation":"Thrown by TFile.Reader.compareKeys(byte[] a, int o1, int l1, byte[] b, int o2, int l2) on an unsorted TFile. Key comparison via the reader delegates to the comparator embedded at write time; unsorted files (writer created with a null comparator name) have no comparator, so the call is rejected with RuntimeException rather than returning a meaningless ordering.","triggerScenarios":"Calling reader.compareKeys(...) to order keys pulled from a TFile that was written without a comparator; common in merge utilities that compare current keys of several readers.","commonSituations":"Merging or diffing TFiles where some inputs were written unsorted; consumers assuming every TFile embeds a comparator; mixed-version pipelines where older writers omitted the comparator.","solutions":["Write input TFiles in sorted mode with an explicit comparator so compareKeys works","For unsorted files, compare raw key bytes yourself (e.g. with Bytes.compareTo or WritableComparator) instead of reader.compareKeys","Branch on reader.isSorted() and use a fallback comparator in the unsorted path"],"exampleFix":"// before\nint c = reader.compareKeys(a, 0, a.length, b, 0, b.length); // RuntimeException if unsorted\n\n// after\nint c = reader.isSorted()\n    ? reader.compareKeys(a, 0, a.length, b, 0, b.length)\n    : Bytes.compareTo(a, 0, a.length, b, 0, b.length);","handlingStrategy":"validation","validationCode":"int compareKeys(TFile.Reader reader, byte[] a, byte[] b) {\n  return reader.isSorted()\n      ? reader.compareKeys(a, 0, a.length, b, 0, b.length)\n      : Bytes.compareTo(a, 0, a.length, b, 0, b.length);\n}","typeGuard":"static boolean hasFileComparator(TFile.Reader reader) {\n  return reader.isSorted();\n}","tryCatchPattern":"catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"unsorted TFiles\")) {\n    // compare raw bytes with your own comparator instead\n  }\n}","preventionTips":["Wrap key comparison in one helper that branches on isSorted()","Ensure merge jobs only pair readers whose comparators match (or all use the same byte comparator)","Prefer writing files sorted with \"memcmp\" for interoperable merges"],"tags":["hadoop","tfile","unsorted","comparator","reader"],"backgroundTag":"requires-sorted-input","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}