apache/flink · error · CliArgsException

Python command line option detected but the flink-python mod

Error message

Python command line option detected but the flink-python module seems to be missing or not working as expected.

What it means

Thrown by ProgramOptionsUtils.createPythonProgramOptions when a Python CLI option (-py, -pyfl, -pym, -pyreq, -pyarch, -pyexec, -pyclientexec, -pp) is detected but the flink-python module (PythonProgramOptions class) cannot be loaded. The method attempts to load the Python jar via a URLClassLoader constructed from PackagedProgramUtils.getPythonJar(), and if Class.forName or constructor invocation fails, it wraps the cause in CliArgsException. This means the Python integration layer is not available.

Source

Thrown at flink-clients/src/main/java/org/apache/flink/client/cli/ProgramOptionsUtils.java:92

                || line.hasOption(PYTHON_PATH.getOpt());
    }

    public static ProgramOptions createPythonProgramOptions(CommandLine line)
            throws CliArgsException {
        try {
            ClassLoader classLoader = getPythonClassLoader();
            Class<?> pythonProgramOptionsClazz =
                    Class.forName(
                            "org.apache.flink.client.cli.PythonProgramOptions", false, classLoader);
            Constructor<?> constructor =
                    pythonProgramOptionsClazz.getConstructor(CommandLine.class);
            return (ProgramOptions) constructor.newInstance(line);
        } catch (InstantiationException
                | InvocationTargetException
                | NoSuchMethodException
                | IllegalAccessException
                | ClassNotFoundException e) {
            throw new CliArgsException(
                    "Python command line option detected but the flink-python module seems to be missing "
                            + "or not working as expected.",
                    e);
        }
    }

    private static ClassLoader getPythonClassLoader() {
        try {
            return new URLClassLoader(
                    new URL[] {PackagedProgramUtils.getPythonJar()},
                    Thread.currentThread().getContextClassLoader());
        } catch (RuntimeException e) {
            LOG.warn(
                    "An attempt to load the flink-python jar from the \"opt\" directory failed, "
                            + "fall back to use the context class loader.",
                    e);
            return Thread.currentThread().getContextClassLoader();
        }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Install the flink-python JAR: copy or symlink flink-dist's opt/flink-python-<version>.jar into the lib/ directory or ensure it's in FLINK_HOME/opt/
2. Verify PyFlink version matches the Flink distribution version exactly
3. Use pip install apache-flink==<matching-version> to get a compatible Python package

Example fix

# before (Python jar missing)
flink run -py my_job.py

# after
# ensure the jar is in place
cp $FLINK_HOME/opt/flink-python-*.jar $FLINK_HOME/lib/
flink run -py my_job.py

// Check Python jar availability before using Python options:
try {
    URL pythonJar = PackagedProgramUtils.getPythonJar();
    if (pythonJar == null || !new File(pythonJar.toURI()).exists()) {
        System.err.println("flink-python JAR not found. Install it in opt/ or lib/.");
    }
} catch (Exception e) {
    System.err.println("Cannot locate flink-python JAR: " + e.getMessage());
}

When it happens

Trigger: Using -py or -pyfl options when the flink-python JAR is not present in FLINK_HOME/opt/ or on the classpath; the JAR exists but is corrupted or version-incompatible; PackagedProgramUtils.getPythonJar() cannot locate the Python jar in the expected directory.

Common situations: Minimal Flink installation without the Python component; version mismatch between the flink-python JAR and the core Flink distribution; custom container images that didn't include the opt/flink-python*.jar.

Related errors

• Bad syntax for classpath: {}
• No cluster id was specified. Please specify a cluster to whi
• The configuration directory '{}', specified in the '{}' envi
• The configuration directory was not specified. Please specif
• No valid command-line found.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/1ce531e17d31e6b8. Report an issue: GitHub.