apache/beam · error · IllegalArgumentException

Value translation's input type is not same as hadoop…

Error message

Value translation's input type is not same as hadoop InputFormat :  %s value class : %s

What it means

HadoopFormatIO.validateTranslationFunction() throws IllegalArgumentException when the user-supplied key/value translation SimpleFunction's input type does not match the expected key or value type of the configured Hadoop InputFormat. It fails fast at pipeline construction rather than at runtime.

Solutions

Make the translation function's input type exactly match the InputFormat's key/value class (e.g. SimpleFunction<LongWritable, Long>).
Verify getinputFormatClass() key/value types via the InputFormat's setInputKeyClass/setInputValueClass configuration and align the function.
Add explicit generic type parameters so the compiler catches the mismatch before runtime.

Example fix

// before
.withValueTranslation(new SimpleFunction<String, String>() {...}) // InputFormat value is Text
// after
.withValueTranslation(new SimpleFunction<Text, String>() {
  public String apply(Text t) { return t.toString(); }
})

Defensive patterns

Strategy: type-guard

Validate before calling

if (!fn.getInputTypeDescriptor().equals(TypeDescriptor.of(keyClass))) { throw new IllegalArgumentException("translation input must be " + keyClass); }

Type guard

<K> SimpleFunction<K,?> typedFn(Class<K> expected, SimpleFunction<? super K,?> fn) { checkArgument(fn.getInputTypeDescriptor().getRawType().isAssignableFrom(expected)); return (SimpleFunction<K,?>) fn; }

Try / catch

try { read.withKeyTranslation(fn).expand(p); } catch (IllegalArgumentException e) { throw new IllegalStateException("Fix translation function input type to match InputFormat key class", e); }

Prevention

Type translation functions with the exact Writable classes from the InputFormat config
Use explicit generics instead of raw lambdas
Validate key/value classes against the function types in tests

When it happens

Trigger: Calling .withKeyTranslation(fn) or .withValueTranslation(fn) on HadoopFormatIO.Read where fn.getInputTypeDescriptor() differs from the InputFormat's key class or value class respectively.

Common situations: Translating a WritableComparable key (e.g. LongWritable) with a function typed for plain Long; copy-pasted translation functions between jobs with different InputFormats; generic type erasure hiding the mismatch until validation.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/088291156af7cbf3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:587

      checkArgument(getConfiguration() != null, "withConfiguration() is required");
      // Validate that the key translation input type must be same as key class of InputFormat.
      validateTranslationFunction(
          getinputFormatKeyClass(),
          getKeyTranslationFunction(),
          "Key translation's input type is not same as hadoop InputFormat : %s key class : %s");
      // Validate that the value translation input type must be same as value class of InputFormat.
      validateTranslationFunction(
          getinputFormatValueClass(),
          getValueTranslationFunction(),
          "Value translation's input type is not same as hadoop InputFormat :  "
              + "%s value class : %s");
    }

    /** Validates translation function given for key/value translation. */
    private void validateTranslationFunction(
        TypeDescriptor<?> inputType, SimpleFunction<?, ?> simpleFunction, String errorMsg) {
      if (simpleFunction != null && !simpleFunction.getInputTypeDescriptor().equals(inputType)) {
        throw new IllegalArgumentException(
            String.format(errorMsg, getinputFormatClass().getRawType(), inputType.getRawType()));
      }
    }

    /**
     * Returns the default coder for a given type descriptor. Coder Registry is queried for correct
     * coder, if not found in Coder Registry, then check if the type descriptor provided is of type
     * Writable, then WritableCoder is returned, else exception is thrown "Cannot find coder".
     */
    @SuppressWarnings({"unchecked", "WeakerAccess"})
    public <T> Coder<T> getDefaultCoder(TypeDescriptor<?> typeDesc, CoderRegistry coderRegistry) {
      Class classType = typeDesc.getRawType();
      try {
        return (Coder<T>) coderRegistry.getCoder(typeDesc);
      } catch (CannotProvideCoderException e) {
        if (Writable.class.isAssignableFrom(classType)) {
          return (Coder<T>) WritableCoder.of(classType);
        }

View on GitHub (pinned to 12126d8942)