{"record":{"id":"6f33d636f9a4f5f3","repo":"apache/hadoop","slug":"no-enough-valid-inputs-are-provided-not-recoverab","errorCode":null,"errorMessage":"No enough valid inputs are provided, not recoverable","messagePattern":"No enough valid inputs are provided, not recoverable","errorType":"exception","errorClass":"HadoopIllegalArgumentException","httpStatus":null,"severity":"critical","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/ByteArrayDecodingState.java","lineNumber":112,"sourceCode":"   */\n  void checkInputBuffers(byte[][] buffers) {\n    int validInputs = 0;\n\n    for (byte[] buffer : buffers) {\n      if (buffer == null) {\n        continue;\n      }\n\n      if (buffer.length != decodeLength) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid buffer, not of length \" + decodeLength);\n      }\n\n      validInputs++;\n    }\n\n    if (validInputs < decoder.getNumDataUnits()) {\n      throw new HadoopIllegalArgumentException(\n          \"No enough valid inputs are provided, not recoverable\");\n    }\n  }\n\n  /**\n   * Check and ensure the buffers are of the desired length.\n   * @param buffers the buffers to check\n   */\n  void checkOutputBuffers(byte[][] buffers) {\n    for (byte[] buffer : buffers) {\n      if (buffer == null) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid buffer found, not allowing null\");\n      }\n\n      if (buffer.length != decodeLength) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid buffer not of length \" + decodeLength);","sourceCodeStart":94,"sourceCodeEnd":130,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/ByteArrayDecodingState.java#L94-L130","documentation":"After validating lengths, ByteArrayDecodingState counts non-null inputs and requires at least decoder.getNumDataUnits() of them. Fewer valid inputs than data units means more erasures than the MDS erasure code can repair, so Hadoop throws HadoopIllegalArgumentException(\"No enough valid inputs are provided, not recoverable\") instead of producing garbage.","triggerScenarios":"Calling decode with more than numParityUnits null entries (all others valid), or with an ErasureCoderOptions whose numDataUnits is larger than the number of readable units actually supplied.","commonSituations":"Multiple concurrent datanode failures beyond policy parity; nulling out failed reads; building decoder options from the wrong schema so unit counts don't match the available inputs.","solutions":["Supply at least numDataUnits non-null, correct-length input buffers","Check that the ErasureCoderOptions (schema) unit counts match the actual block group layout before decoding","If failures genuinely exceed parity, treat the group as unrecoverable and fall back to replica/backup recovery — no API call can repair it"],"exampleFix":"// before\nbyte[][] inputs = new byte[numTotalUnits][]; // 4 nulls with 6+3 policy\ndecoder.decode(inputs, erased, outputs); // throws: not recoverable\n\n// before decode: count readable units\nlong valid = Arrays.stream(inputs).filter(Objects::nonNull).count();\nif (valid < decoder.getNumDataUnits()) {\n  throw new IOException(\"Unrecoverable: only \" + valid + \" valid units\");\n}","handlingStrategy":"validation","validationCode":"int valid = 0;\nfor (byte[] b : inputs) if (b != null) valid++;\nif (valid < decoder.getNumDataUnits()) {\n  throw new IOException(\"Cannot recover: \" + valid + \" valid units < \"\n      + decoder.getNumDataUnits() + \" data units; parity exhausted\");\n}\ndecoder.decode(inputs, erased, outputs);","typeGuard":"boolean isRecoverable(Object[] inputs, int numDataUnits) {\n  int n = 0;\n  for (Object in : inputs) if (in != null && ++n >= numDataUnits) return true;\n  return false;\n}","tryCatchPattern":"try {\n  decoder.decode(inputs, erased, outputs);\n} catch (HadoopIllegalArgumentException e) {\n  if (e.getMessage().contains(\"not recoverable\")) {\n    // permanently failed group: alert, stop retrying, use backup/replica path\n  }\n}","preventionTips":["Before decoding, compare live units against numDataUnits of the policy schema","Don't retry unrecoverable decodes — more erasures than parity is mathematically hopeless","Monitor datanode failure counts against each EC policy's parity limit"],"tags":["erasure-coding","unrecoverable","validation"],"backgroundTag":"too-many-erasures","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}