apache/iceberg · error · IOException

Unrecognized version or corrupt state: ${version}

Error message

Unrecognized version or corrupt state: ${version}

What it means

IcebergCommittableSerializer.deserialize validates the version byte/int of a serialized IcebergCommittable; anything other than VERSION=1 (the only supported format) throws an IOException 'Unrecognized version or corrupt state'. This guards against deserializing checkpoint state or network payloads written by an incompatible serializer version or a corrupted byte stream.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergCommittableSerializer.java:66

    view.writeInt(committable.manifest().length);
    view.write(committable.manifest());
    return out.toByteArray();
  }

  @Override
  public IcebergCommittable deserialize(int version, byte[] serialized) throws IOException {
    if (version == 1) {
      DataInputDeserializer view = new DataInputDeserializer(serialized);
      String jobId = view.readUTF();
      String operatorId = view.readUTF();
      long checkpointId = view.readLong();
      int manifestLen = view.readInt();
      byte[] manifestBuf;
      manifestBuf = new byte[manifestLen];
      view.read(manifestBuf);
      return new IcebergCommittable(manifestBuf, jobId, operatorId, checkpointId);
    }
    throw new IOException("Unrecognized version or corrupt state: " + version);
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Resume the job with the same iceberg-flink-runtime version that produced the checkpoint/savepoint
2. Discard the incompatible state: restart without savepoint or use --allowNonRestoredState after confirming no committables are lost mid-checkpoint
3. If a version upgrade is required, drain the pipeline (let the checkpoint commit) before upgrading
4. Check for duplicate Iceberg jars on the classpath causing the wrong serializer to read the bytes

Example fix

// before
./bin/flink run -s flink-savepoint-13t ... (savepoint made with iceberg 1.4, running 1.5 runtime)
// after
# run with matching runtime, or start fresh:
./bin/flink run -s flink-savepoint-13t --jarfile iceberg-flink-runtime-1.4.3.jar ...

try {
  IcebergCommittable c = serializer.deserialize(version, bytes);
} catch (IOException e) {
  if (e.getMessage().startsWith("Unrecognized version")) {
    // incompatible savepoint/format: fail fast, do not retry
    throw new StateIncompatibleException(e);
  }
  throw e;
}

When it happens

Trigger: Restoring a Flink job from a savepoint/checkpoint written by a different Iceberg version whose committable format differs, or deserializing a byte array that was truncated/garbled (corrupt state) so the leading version marker does not equal 1.

Common situations: Upgrading or downgrading iceberg-flink-runtime between job restarts while resuming from old state; manually crafting IcebergCommittable bytes; classpath mixing two serializer implementations.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Unrecognized version or corrupt state: ${version}
• Unrecognized version or corrupt state: <version>
• Unknown serialize version: ${version}
• Could not deserialize the WriteResult object
• Unrecognized version or corrupt state: ${version}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/bf5a0ac5f0be9621. Report an issue: GitHub.