apache/flink · error · CliArgsException

Could not get job jar and dependencies from JAR file: {}

Error message

Could not get job jar and dependencies from JAR file: {}

What it means

Thrown from AbstractColumnReader.prepareNewPage when a page is dictionary-encoded but the concrete encoding is neither PLAIN_DICTIONARY nor RLE_DICTIONARY. Flink's vectorized reader only implements these two dictionary encodings for page data. Any other encoding claiming to use a dictionary (now or in the future) is rejected.

Source

Thrown at flink-clients/src/main/java/org/apache/flink/client/cli/CliFrontend.java:259

                                        new IllegalArgumentException(
                                                String.format(
                                                        "Config '%s' has to be set.",
                                                        DeploymentOptions.TARGET.key())));

        return executionTarget.trim().endsWith("application");
    }

    /** Get all provided libraries needed to run the program from the ProgramOptions. */
    private List<URL> getJobJarAndDependencies(ProgramOptions programOptions)
            throws CliArgsException {
        String entryPointClass = programOptions.getEntryPointClassName();
        String jarFilePath = programOptions.getJarFilePath();

        try {
            File jarFile = jarFilePath != null ? getJarFile(jarFilePath) : null;
            return PackagedProgram.getJobJarAndDependencies(jarFile, entryPointClass);
        } catch (FileNotFoundException | ProgramInvocationException e) {
            throw new CliArgsException(
                    "Could not get job jar and dependencies from JAR file: " + e.getMessage(), e);
        }
    }

    private PackagedProgram getPackagedProgram(
            ProgramOptions programOptions, Configuration effectiveConfiguration)
            throws ProgramInvocationException, CliArgsException {
        PackagedProgram program;
        try {
            LOG.info("Building program from JAR file");
            program = buildProgram(programOptions, effectiveConfiguration);
        } catch (FileNotFoundException e) {
            throw new CliArgsException(
                    "Could not build the program from JAR file: " + e.getMessage(), e);
        }
        return program;
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Identify the encoding: the message includes it (e.g. 'Unsupported encoding: DELTA_BYTE_SPLIT')
2. Rewrite the file with a writer using standard encodings (parquet-mr defaults) — e.g. in Spark set spark.sql.parquet.compression.codec and disable experimental encoding options
3. Upgrade Flink to a release whose parquet-mr supports the encoding
4. Fall back to a non-vectorized Parquet reading path (e.g. Hive/MapReduce-based InputFormat) that delegates to parquet-mr's full ValuesReader set

// Pre-check encodings against the supported set before vectorized read
Set<Encoding> supported = Set.of(Encoding.PLAIN, Encoding.PLAIN_DICTIONARY, Encoding.RLE_DICTIONARY);
try (ParquetFileReader r = ParquetFileReader.open(conf, path)) {
    for (BlockMetaData b : r.getFooter().getBlocks()) {
        for (ColumnChunkMetaData c : b.getColumns()) {
            for (Encoding enc : c.getEncodings()) {
                if (!supported.contains(enc)) throw new UnsupportedEncodingException(enc.name());
            }
        }
    }
}

try {
    reader.readToVector(...);
} catch (IOException | UnsupportedOperationException e) {
    if (String.valueOf(e.getMessage()).startsWith("Unsupported encoding")) {
        // route file to a converter or reject with a clear producer-facing message
    } else throw e;
}

When it happens

Trigger: A Parquet file written with a new/experimental dictionary encoding not in {PLAIN_DICTIONARY, RLE_DICTIONARY}; enum Encoding.usesDictionary() returning true for an encoding Flink's vectorized path never anticipated.

Common situations: Reading files from newer writers that adopted encodings Flink's vendored Parquet version does not know; files produced by research/experimental Parquet implementations; after upgrading a writer stack that switched encodings by default.

Related errors

• Failed to retrieve job list.
• Error while waiting for job to be initialized
• Translator {} cannot translate the given pipeline {}.
• Could not stop with a savepoint job "{}".
• could not decode the dictionary for {}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/e052e3e5b1dcd9eb. Report an issue: GitHub.