apache/flink · error · CompletionException

Failed to serialize ExecutionPlan.

Error message

Failed to serialize ExecutionPlan.

What it means

Thrown when RestClusterClient.submitJob fails to serialize the ExecutionPlan to a temporary file via Java ObjectOutputStream. The ExecutionPlan (containing the job graph, user jars, and artifacts) must be written to a .bin file that is then uploaded to the JobManager REST endpoint. This wraps the underlying IOException (disk failure, non-serializable object, or permission error).

Source

Thrown at flink-clients/src/main/java/org/apache/flink/client/program/rest/RestClusterClient.java:384

    @Override
    public CompletableFuture<JobID> submitJob(@Nonnull ExecutionPlan executionPlan) {
        CompletableFuture<java.nio.file.Path> executionPlanFileFuture =
                CompletableFuture.supplyAsync(
                        () -> {
                            try {
                                final java.nio.file.Path executionPlanFile =
                                        Files.createTempFile(
                                                "flink-executionPlan-" + executionPlan.getJobID(),
                                                ".bin");
                                try (ObjectOutputStream objectOut =
                                        new ObjectOutputStream(
                                                Files.newOutputStream(executionPlanFile))) {
                                    objectOut.writeObject(executionPlan);
                                }
                                return executionPlanFile;
                            } catch (IOException e) {
                                throw new CompletionException(
                                        new FlinkException(
                                                "Failed to serialize ExecutionPlan.", e));
                            }
                        },
                        executorService);

        CompletableFuture<Tuple2<JobSubmitRequestBody, Collection<FileUpload>>> requestFuture =
                executionPlanFileFuture.thenApply(
                        executionPlanFile -> {
                            List<String> jarFileNames = new ArrayList<>(8);
                            List<JobSubmitRequestBody.DistributedCacheFile> artifactFileNames =
                                    new ArrayList<>(8);
                            Collection<FileUpload> filesToUpload = new ArrayList<>(8);

                            filesToUpload.add(
                                    new FileUpload(
                                            executionPlanFile, RestConstants.CONTENT_TYPE_BINARY));

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Check the wrapped IOException cause in the stack trace — if it is NotSerializableException, find and mark the offending field as transient or make the enclosing class implement Serializable.
2. Verify the client's temp directory is writable and has free space: check java.io.tmpdir and disk usage.
3. Ensure all user jar paths and artifact paths in the ExecutionPlan are accessible from the client's filesystem.
4. If the error is transient (disk full), free disk space and retry job submission.

Example fix

// before — non-serializable field in a UDF
public class MyMapper extends RichMapFunction<String,String> {
    private Connection dbConn; // not serializable!
}
// after — open in open() method, mark transient
public class MyMapper extends RichMapFunction<String,String> {
    private transient Connection dbConn;
    @Override
    public void open(Configuration cfg) { dbConn = DriverManager.getConnection(...); }
}

// Validate ExecutionPlan is serializable before submission
try (ByteArrayOutputStream bos = new ByteArrayOutputStream();
     ObjectOutputStream oos = new ObjectOutputStream(bos)) {
    oos.writeObject(executionPlan);
} catch (IOException e) {
    throw new IllegalArgumentException("ExecutionPlan is not serializable: " + e.getMessage(), e);
}

try {
    client.submitJob(executionPlan).get(30, TimeUnit.SECONDS);
} catch (ExecutionException e) {
    Throwable cause = ExceptionUtils.stripExecutionException(e);
    if (cause instanceof FlinkException && cause.getMessage().contains("Failed to serialize")) {
        // handle serialization failure — inspect NotSerializableException in cause.getCause()
    }
}

When it happens

Trigger: Calling submitJob(ExecutionPlan) where the ExecutionPlan graph contains a non-Serializable object; the system temp directory (java.io.tmpdir) is full or not writable; the ObjectOutputStream.writeObject(executionPlan) call fails because an element in the plan's user jars or graph is not serializable.

Common situations: User job JAR contains a non-serializable UDF field (e.g., a raw database Connection or Thread object stored in an operator); running in a container with a read-only /tmp; disk pressure on the client machine; classpath mismatch where an object referenced by the ExecutionPlan is from an incompatible classloader.

Related errors

• Failed to get the FileSystem of artifact {artifactFilePath}.
• Failed to submit ExecutionPlan.
• Failed to deserialize coordination response
• Cannot deserialize and unwrap accumulators properly.
• Unable to deserialize message

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/407c11e74ce7a5cf. Report an issue: GitHub.