{"record":{"id":"6b5b25fafff34ccb","repo":"apache/flink","slug":"error-while-waiting-for-job-to-be-initialized","errorCode":null,"errorMessage":"Error while waiting for job to be initialized","messagePattern":"Error while waiting for job to be initialized","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"flink-clients/src/main/java/org/apache/flink/client/ClientUtils.java","lineNumber":185,"sourceCode":"            while (status == JobStatus.INITIALIZING) {\n                Thread.sleep(waitStrategy.sleepTime(attempt++));\n                status = jobStatusSupplier.get();\n            }\n            if (status == JobStatus.FAILED) {\n                JobResult result = jobResultSupplier.get();\n                Optional<SerializedThrowable> throwable = result.getSerializedThrowable();\n                if (throwable.isPresent()) {\n                    Throwable t = throwable.get().deserializeError(userCodeClassloader);\n                    if (t instanceof JobInitializationException) {\n                        throw t;\n                    }\n                }\n            }\n        } catch (JobInitializationException initializationException) {\n            throw initializationException;\n        } catch (Throwable throwable) {\n            ExceptionUtils.checkInterrupted(throwable);\n            throw new RuntimeException(\"Error while waiting for job to be initialized\", throwable);\n        }\n    }\n\n    /**\n     * The client reports the heartbeat to the dispatcher for aliveness.\n     *\n     * @param jobClient The job client.\n     * @param interval The heartbeat interval.\n     * @param timeout The heartbeat timeout.\n     * @return The ScheduledExecutorService which reports heartbeat periodically.\n     */\n    public static ScheduledExecutorService reportHeartbeatPeriodically(\n            JobClient jobClient, long interval, long timeout) {\n        checkArgument(\n                interval < timeout,\n                \"The client's heartbeat interval \"\n                        + \"should be less than the heartbeat timeout. Please adjust the param '\"\n                        + ClientOptions.CLIENT_HEARTBEAT_INTERVAL","sourceCodeStart":167,"sourceCodeEnd":203,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-clients/src/main/java/org/apache/flink/client/ClientUtils.java#L167-L203","documentation":"Thrown while initializing a Parquet DataPageV2 for a column in Flink's vectorized Parquet reader (AbstractColumnReader.readPageV2). The wrapper IOException means prepareNewPage failed while setting up the data decoder for the page body — the cause (nested exception) carries the real reason, such as a truncated stream, an unsupported encoding, or a missing dictionary. It indicates the column chunk's page data cannot be decoded with the page's declared data encoding.","triggerScenarios":"Reading a Parquet file whose DataPageV2 body is corrupt or truncated; the page declares an encoding prepareNewPage rejects (see the dictionary-missing and unsupported-encoding branches it calls); a dictionary-encoded page arriving when the dictionary page was never read for that column chunk; partial file writes or bad transfers producing short page byte arrays.","commonSituations":"Ingesting Parquet files produced by non-standard writers or writers with experimental v2 pages; files corrupted in object-store transfer or truncated uploads; files where column chunks are written with encodings Flink's vectorized reader does not implement; upgrading a writer library that changed default page encodings.","solutions":["Inspect the nested cause exception — the fix depends entirely on whether it is 'dictionary was missing', 'Unsupported encoding', or a stream EOF","Verify file integrity: re-download/regenerate the file and compare length/CRC; run `parquet-tools meta/dump` on it to confirm page structure is valid","If the cause is an unsupported encoding, rewrite the file with a standard writer (e.g. parquet-mr defaults: PLAIN / PLAIN_DICTIONARY / RLE_DICTIONARY) or read it through a path that does not use the vectorized reader","Reproduce with the same file in parquet-mr's VerifyAndWrite / dump tool to determine whether the defect is in the file or in Flink's reader"],"exampleFix":"// No code fix — data-side issue.\n// Diagnose with the cause:\n} catch (IOException e) {\n    Throwable root = ExceptionUtils.getRootCause(e);\n    // root tells you: dictionary missing / unsupported encoding / EOF\n}","handlingStrategy":"try-catch","validationCode":"// Before reading, sanity-check page structure with parquet-mr metadata\nParquetFileReader reader = ParquetFileReader.open(conf, path);\nParquetMetadata md = reader.getFooter();\nfor (ColumnChunkMetaData cc : md.getBlocks().get(0).getColumns()) {\n    // pages for the chunk can be inspected via reader.readNextChunk()/getPageIndex\n    // reject files whose encodings are unknown to Encoding.values()\n}","typeGuard":null,"tryCatchPattern":"try {\n    vectorizedReader.readToVector(n, vector);\n} catch (IOException e) {\n    // 'could not read page ... in col ...' — inspect root cause before retry\n    Throwable cause = ExceptionUtils.getRootCause(e);\n    LOG.error(\"Parquet page decode failed for {}\", descriptor, cause);\n    throw e; // never blind-retry: corruption is deterministic per file\n}","preventionTips":["Validate Parquet files (footer + pages) with parquet-tools at ingestion time","Use checksummed/end-to-end verified transfers to object storage","Keep the Flink parquet format's vendored parquet-mr version >= the versions used by your writers"],"tags":["parquet","io","encoding","data-corruption"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}