apache/hadoop · error · IllegalArgumentException

Unsupported comparator: {comparator}

Error message

Unsupported comparator: {comparator}

What it means

IllegalArgumentException from makeComparator when the comparator string is non-empty, is not "memcmp", and does not start with "jclass:". Only three shapes are legal: "" (unsorted), "memcmp" (default byte order), and "jclass:<fqcn>". The string is persisted in the file, so an invalid name written at creation time also explodes on every subsequent read.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java:2122

        }
        String compClassName =
            comparator.substring(COMPARATOR_JCLASS.length()).trim();
        try {
          // Resolve without running the class initializer, confirm it really
          // is a RawComparator, and only then load and construct it.
          Class<?> compClass =
              Class.forName(compClassName, false, conf.getClassLoader());
          RawComparator<Object> rawComparator =
              (RawComparator<Object>) compClass.asSubclass(RawComparator.class)
                  .getDeclaredConstructor().newInstance();
          return new BytesComparator(rawComparator);
        } catch (Exception e) {
          throw new IllegalArgumentException(
              "Failed to instantiate comparator: " + comparator + "("
                  + e.toString() + ")");
        }
      } else {
        throw new IllegalArgumentException("Unsupported comparator: "
            + comparator);
      }
    }

    public void write(DataOutput out) throws IOException {
      TFile.API_VERSION.write(out);
      Utils.writeVLong(out, recordCount);
      Utils.writeString(out, strComparator);
    }

    public long getRecordCount() {
      return recordCount;
    }

    public void incRecordCount() {
      ++recordCount;
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Use the constants: TFile.COMPARATOR_MEMCMP ("memcmp") or TFile.COMPARATOR_JCLASS ("jclass:") concatenated with the class name.
2. Pass null or empty string for unsorted files instead of a descriptive label.
3. Validate the value before constructing the Writer: it must be "", "memcmp", or start with "jclass:".

Example fix

// before
TFile.Writer w = new TFile.Writer(out, minBlockSize, "MyComparator", conf);
// after
String cmp = TFile.COMPARATOR_MEMCMP; // or TFile.COMPARATOR_JCLASS + MyKeyComparator.class.getName()
TFile.Writer w = new TFile.Writer(out, minBlockSize, cmp, conf);

static String validComparator(String s) {
  if (s == null || s.isEmpty()) return ""; // unsorted
  if (TFile.COMPARATOR_MEMCMP.equals(s)) return s;
  if (s.startsWith(TFile.COMPARATOR_JCLASS)) return s;
  throw new IllegalArgumentException("Use TFile.COMPARATOR_MEMCMP or 'jclass:<fqcn>': " + s);
}
TFile.Writer w = new TFile.Writer(out, minBlockSize, validComparator(name), conf);

When it happens

Trigger: Passing a hand-typed comparator name to new TFile.Writer(out, blockSize, comparatorName, conf): typos like "jclas:", "Memcmp", "memcmp ", or a fully-qualified class name without the "jclass:" prefix. Also fires at read time for legacy files carrying arbitrary comparator names that older code tolerated.

Common situations: Config-driven comparator names where a typo survives to production, migration from code that treated the parameter as a free-form class name, or case/whitespace slips in generated job configs.

Related errors

• Class-name comparators are not enabled (set tfile.comparator
• Failed to instantiate comparator: {comparator}({e})
• Unsupported block buffer "{}"
• Errors on getting mount table loader class. The fs.viewfs.mo
• %s=null: %s: %s

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5968457c63de7416. Report an issue: GitHub.