{"record":{"id":"dd084077974e73a7","repo":"apache/hadoop","slug":"entries-are-not-comparable-for-unsorted-tfiles","errorCode":null,"errorMessage":"Entries are not comparable for unsorted TFiles","messagePattern":"Entries are not comparable for unsorted TFiles","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java","lineNumber":937,"sourceCode":"     * @return The last key in the TFile.\n     * @throws IOException raised on errors performing I/O.\n     */\n    public RawComparable getLastKey() throws IOException {\n      checkTFileDataIndex();\n      return tfileIndex.getLastKey();\n    }\n\n    /**\n     * Get a Comparator object to compare Entries. It is useful when you want\n     * stores the entries in a collection (such as PriorityQueue) and perform\n     * sorting or comparison among entries based on the keys without copying out\n     * the key.\n     * \n     * @return An Entry Comparator..\n     */\n    public Comparator<Scanner.Entry> getEntryComparator() {\n      if (!isSorted()) {\n        throw new RuntimeException(\n            \"Entries are not comparable for unsorted TFiles\");\n      }\n\n      return new Comparator<Scanner.Entry>() {\n        /**\n         * Provide a customized comparator for Entries. This is useful if we\n         * have a collection of Entry objects. However, if the Entry objects\n         * come from different TFiles, users must ensure that those TFiles share\n         * the same RawComparator.\n         */\n        @Override\n        public int compare(Scanner.Entry o1, Scanner.Entry o2) {\n          return comparator.compare(o1.getKeyBuffer(), 0, o1.getKeyLength(), o2\n              .getKeyBuffer(), 0, o2.getKeyLength());\n        }\n      };\n    }\n","sourceCodeStart":919,"sourceCodeEnd":955,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java#L919-L955","documentation":"Thrown by TFile.Reader.getEntryComparator() when the TFile was written without a comparator (unsorted mode, writer created with a null comparator name). An entry comparator only exists for sorted TFiles, because comparing entries means comparing keys with the file's embedded comparator. Unsorted files carry none, so the request is rejected with RuntimeException.","triggerScenarios":"Calling reader.getEntryComparator() on a TFile written with new TFile.Writer(out, size, compressName, null); typical when putting Scanner.Entry objects into a PriorityQueue or TreeSet to merge or top-N entries.","commonSituations":"Generic merge code that assumes all TFiles are sorted; switching a producer from sorted to unsorted output while the consumer still collects entries into ordered collections; comparing entries across files that were written with different configurations.","solutions":["Write the TFile in sorted mode by passing a comparator name (e.g. \"memcmp\" or a custom RawComparator class name) to the TFile.Writer constructor","If the file must stay unsorted, sort entries externally with your own comparator over the raw key bytes (entry.compareTo other entries is not defined)","Guard with reader.isSorted() before calling getEntryComparator() and fail with a clear domain error otherwise"],"exampleFix":"// before\nComparator<Scanner.Entry> cmp = reader.getEntryComparator(); // RuntimeException on unsorted file\n\n// after\nif (!reader.isSorted()) {\n  throw new IOException(\"Cannot merge: input TFile is not sorted: \" + path);\n}\nComparator<Scanner.Entry> cmp = reader.getEntryComparator();","handlingStrategy":"validation","validationCode":"if (!reader.isSorted()) {\n  throw new IOException(\"getEntryComparator requires a sorted TFile: \" + path);\n}\nComparator<Scanner.Entry> entryCmp = reader.getEntryComparator();","typeGuard":"static boolean supportsEntryComparison(TFile.Reader reader) {\n  return reader.isSorted();\n}","tryCatchPattern":"catch (RuntimeException e) {\n  if (\"Entries are not comparable for unsorted TFiles\".equals(e.getMessage())) {\n    // fall back to external sorting with an application comparator over raw key bytes\n  }\n}","preventionTips":["Standardize producer writers on sorted mode (explicit comparator name) when consumers merge entries","Check reader.isSorted() before any comparator-dependent API","Document per dataset whether files are sorted and with which comparator"],"tags":["hadoop","tfile","unsorted","comparator","reader"],"backgroundTag":"requires-sorted-input","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}