apache/iceberg · error · UncheckedIOException

Failed to decode partition

Error message

Failed to decode partition

What it means

StructLikeSerializer.decodePartition reads back bytes produced by encodePartition into a new StructLike; an IOException while reading is wrapped in an UncheckedIOException. This usually means the byte array is truncated, corrupt, or was not produced by encodePartition with the same partition type.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/StructLikeSerializer.java:121

        boolean isNull = dis.readBoolean();
        if (isNull) {
          partition.set(i, null);
        } else {
          int length = dis.readInt();
          byte[] bytes = new byte[length];
          dis.readFully(bytes);
          Object value = Conversions.fromByteBuffer(fields.get(i).type(), ByteBuffer.wrap(bytes));
          // Conversions returns CharBuffer for STRING; PartitionData (and the manifest writer)
          // expects String.
          if (value instanceof CharSequence cs && !(value instanceof String)) {
            value = cs.toString();
          }

          partition.set(i, value);
        }
      }
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to decode partition", e);
    }

    return partition;
  }

  private void writeField(StructLike struct, int pos, Type fieldType) throws IOException {
    Object value = struct.get(pos, Object.class);
    if (value == null) {
      dos.writeBoolean(true);
      return;
    }

    dos.writeBoolean(false);
    ByteBuffer buf = Conversions.toByteBuffer(fieldType, value);
    dos.writeInt(buf.remaining());
    dos.write(buf.array(), buf.arrayOffset() + buf.position(), buf.remaining());
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Ensure the partitionType passed to decodePartition matches the one used to encode the bytes.
2. Check for schema/partition spec evolution between when state was written and read back.
3. If state is corrupt, restore from an earlier checkpoint or rebuild state.

Example fix

// before
StructLike p = StructLikeSerializer.decodePartition(bytes, currentSpec.partitionType());
// after
// use the spec captured when the bytes were encoded
StructLike p = StructLikeSerializer.decodePartition(bytes, specAtEncodeTime.partitionType());

// Verify bytes length and partition type match what was used at encode time before decoding

try { StructLike p = StructLikeSerializer.decodePartition(bytes, spec.partitionType()); } catch (UncheckedIOException e) { LOG.error("Partition decode failed; spec mismatch or corrupt bytes", e); }

When it happens

Trigger: Calling decodePartition with bytes that are malformed, truncated, or encoded against a different/older partition type (field count or type mismatch while reading fields).

Common situations: State restored across schema evolution (partition spec changed), corrupted keyed state in checkpoints, or manually constructed byte arrays.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Altering partition keys is not supported yet.
• Failed to encode partition
• Failed to deserialize sort key sketch
• Fail to deserialize data statistics
• Fail to deserialize aggregated statistics,change to v1

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/be1ef28aa91dfa25. Report an issue: GitHub.