What causes "Invalid inputs are found, all being null" in apache/hadoop?

Before decoding, verify at least numDataUnits inputs are non-null and readable; if not, the group genuinely cannot be recovered by EC alone

apache/hadoop · error · HadoopIllegalArgumentException

Invalid inputs are found, all being null

Error message

Invalid inputs are found, all being null

What it means

HHUtil.findFirstValidInput scans an input array for the first non-null element and is used by Hitchhiker coding steps to infer the working buffer size; if every element is null it throws HadoopIllegalArgumentException("Invalid inputs are found, all being null"). Decoding needs at least one readable input to proceed at all.

Solutions

Before decoding, verify at least numDataUnits inputs are non-null and readable; if not, the group genuinely cannot be recovered by EC alone
Propagate read failures as errors rather than silently replacing chunks with null
Pass erasure information via erasedIndexes/erasedMasks while keeping surviving units' buffers non-null

Example fix

// before
for (int i = 0; i < inputs.length; i++) {
  if (readFailed[i]) inputs[i] = null; // may null out everything
}
step.performCoding(inputChunks, outputChunks); // throws: all being null

// after
if (countNonNull(inputs) < numDataUnits) {
  throw new IOException("Block group unrecoverable: fewer than "
      + numDataUnits + " readable units");
}

Defensive patterns

Strategy: validation

Validate before calling

int nonNull = 0;
for (ByteBuffer b : inputs) if (b != null) nonNull++;
if (nonNull == 0) {
  throw new IOException("Cannot decode: no readable input units");
}
if (nonNull < decoder.getNumDataUnits()) {
  throw new IOException("Unrecoverable: only " + nonNull + " of "
      + decoder.getNumDataUnits() + " needed units readable");
}

Type guard

boolean hasEnoughValidInputs(Object[] inputs, int required) {
  int n = 0;
  for (Object in : inputs) if (in != null && ++n >= required) return true;
  return false;
}

Try / catch

try {
  step.performCoding(in, out);
} catch (HadoopIllegalArgumentException e) {
  if (e.getMessage().contains("all being null")) {
    // treat as unrecoverable group: report, do not retry decode
  }
}

Prevention

Never silently substitute null for failed chunk reads; fail or retry the read first
Check readable-unit count against numDataUnits before every decode
Surface per-unit read errors so operators see which datanodes failed

When it happens

Trigger: Calling HH decode/encode paths with an inputs array whose elements are all null — typically every unit is missing/erased, or the caller represented upstream read failures as nulls and lost all of them. Also hit when an empty-capability buffer array (all slots null) is passed to CoderUtil/HHUtil.findFirstValidInput via raw coder state construction.

Common situations: More concurrent failures than the EC policy's parity can tolerate; datanode read errors swallowed and turned into null inputs; logic errors that null out inputs before decoding instead of passing erased indexes.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8f1e8946f1b6b04f. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/util/HHUtil.java:217

    return piggybacks;
  }

  /**
   * Find the valid input from all the inputs.
   *
   * @param <T> Generics Type T.
   * @param inputs input buffers to look for valid input
   * @return the first valid input
   */
  public static <T> T findFirstValidInput(T[] inputs) {
    for (T input : inputs) {
      if (input != null) {
        return input;
      }
    }

    throw new HadoopIllegalArgumentException(
            "Invalid inputs are found, all being null");
  }
}

View on GitHub (pinned to 2add963021)