apache/iceberg · error · UncheckedIOException

Failed to decode partition

Error message

Failed to decode partition

What it means

StructLikeSerializer.decodePartition reads a previously encoded partition from bytes and sets values on a partition StructLike; IOExceptions are wrapped in UncheckedIOException with this message. It occurs when the byte array does not match the expected partition-type layout, i.e. corrupt or version-mismatched encoded data.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/StructLikeSerializer.java:121

        boolean isNull = dis.readBoolean();
        if (isNull) {
          partition.set(i, null);
        } else {
          int length = dis.readInt();
          byte[] bytes = new byte[length];
          dis.readFully(bytes);
          Object value = Conversions.fromByteBuffer(fields.get(i).type(), ByteBuffer.wrap(bytes));
          // Conversions returns CharBuffer for STRING; PartitionData (and the manifest writer)
          // expects String.
          if (value instanceof CharSequence cs && !(value instanceof String)) {
            value = cs.toString();
          }

          partition.set(i, value);
        }
      }
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to decode partition", e);
    }

    return partition;
  }

  private void writeField(StructLike struct, int pos, Type fieldType) throws IOException {
    Object value = struct.get(pos, Object.class);
    if (value == null) {
      dos.writeBoolean(true);
      return;
    }

    dos.writeBoolean(false);
    ByteBuffer buf = Conversions.toByteBuffer(fieldType, value);
    dos.writeInt(buf.remaining());
    dos.write(buf.array(), buf.arrayOffset() + buf.position(), buf.remaining());
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Ensure the partition type used for decoding matches the one used at encoding time.
2. Restore from a checkpoint created with the same Iceberg version and unchanged partition spec.
3. If partition spec evolved, reprocess from scratch instead of restoring old encoded state.

// Confirm the partition type used to encode matches the one used to decode:
if (!partitionType.equals(encodedAgainstType)) {
  throw new IllegalStateException("Partition type mismatch between encode and decode");
}

try {
  partition = StructLikeSerializer.decodePartition(bytes, partitionType);
} catch (UncheckedIOException e) {
  LOG.error("Corrupt or version-mismatched partition payload", e);
  throw e;
}

When it happens

Trigger: decodePartition called with bytes that were not produced by encodePartition for the same partition type — e.g. restored operator state written by a different schema/version, or corrupted index payload.

Common situations: Restoring checkpoints across schema evolution (partition spec changed), or cross-version state restore where the encoded layout changed.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Failed to deserialize sort key sketch
• Fail to deserialize data statistics
• Fail to deserialize aggregated statistics
• Failed to deserialize sort key sketch
• Fail to deserialize data statistics

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/6d59818b3eead6c7. Report an issue: GitHub.