apache/hadoop · error · IOException

encoded array component type {} is not a candidate primitive

Error message

encoded array component type {} is not a candidate primitive type

What it means

During deserialization, readFields() reads the component type name as a UTF8 string and maps it through PRIMITIVE_NAMES. If the name is not one of the eight primitive names, the stream cannot be decoded as a primitive array and an IOException is thrown. This indicates the bytes were not written by ArrayPrimitiveWritable.write(), or the stream is corrupt/misaligned.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ArrayPrimitiveWritable.java:213

      writeDoubleArray(out);
    } else {
      throw new IOException("Component type " + componentType.toString()
          + " is set as the output type, but no encoding is implemented for this type.");
    }
  }

  /* 
   * @see org.apache.hadoop.io.Writable#readFields(java.io.DataInput)
   */
  @Override
  public void readFields(DataInput in) throws IOException {
    
    // read and set the component type of the array
    @SuppressWarnings("deprecation")
    String className = UTF8.readString(in);
    Class<?> componentType = getPrimitiveClass(className);
    if (componentType == null) {
      throw new IOException("encoded array component type "
          + className + " is not a candidate primitive type");
    }
    checkDeclaredComponentType(componentType);
    this.componentType = componentType;
  
    // read and set the length of the array
    int length = in.readInt();
    if (length < 0) {
      throw new IOException("encoded array length is negative " + length);
    }
    this.length = length;
    
    // construct and read in the array
    value = Array.newInstance(componentType, length);

    // do the inner loop.  Walk the decision tree only once.
    if (componentType == Boolean.TYPE) {             // boolean
      readBooleanArray(in);

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the reader and writer use the same Writable class and Hadoop version for that field.
2. Re-generate or migrate the data after changing a serialized field's type.
3. If parsing raw buffers, confirm the offset/length you pass to the DataInput matches where write() started.
4. Wrap readFields in IOException handling and quarantine the bad record instead of failing the whole task.

try {
  w.readFields(in);
} catch (IOException e) {
  LOG.warn("Corrupt or mismatched record at offset " + offset, e);
  // skip/quarantine record, or fail the task depending on tolerance
}

When it happens

Trigger: Feeding bytes written by a different Writable (e.g. ArrayWritable or a custom type) into readFields(); reading at a wrong offset in a hand-managed buffer; truncation or garbage input; a version change where the wire payload for a field changed type.

Common situations: Changing a field's writable class between job versions while old sequence/RPC data is still read; manually slicing shuffle/RPC bytes; endianness or double-read of the same stream position.

Related errors

• encoded array length is negative {}
• No URI in deserialized Path
• Encoded type {} converted to valid component type {} but no
• Cannot initialize the class: {clazz}
• tried to deserialize {} bytes of data! newLength must be no

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/885b25cfbb09da11. Report an issue: GitHub.