apache/flink · error · CliArgsException

Python command line option detected but the flink-python…

Error message

Python command line option detected but the flink-python module seems to be missing or not working as expected.

What it means

Thrown by ProgramOptionsUtils.createPythonProgramOptions when a Python CLI option (-py, -pyfl, -pym, -pyreq, -pyarch, -pyexec, -pyclientexec, -pp) is detected but the flink-python module (PythonProgramOptions class) cannot be loaded. The method attempts to load the Python jar via a URLClassLoader constructed from PackagedProgramUtils.getPythonJar(), and if Class.forName or constructor invocation fails, it wraps the cause in CliArgsException. This means the Python integration layer is not available.

Solutions

1. Install the flink-python JAR: copy or symlink flink-dist's opt/flink-python-<version>.jar into the lib/ directory or ensure it's in FLINK_HOME/opt/
2. Verify PyFlink version matches the Flink distribution version exactly
3. Use pip install apache-flink==<matching-version> to get a compatible Python package

Example fix

# before (Python jar missing)
flink run -py my_job.py

# after
# ensure the jar is in place
cp $FLINK_HOME/opt/flink-python-*.jar $FLINK_HOME/lib/
flink run -py my_job.py

// Check Python jar availability before using Python options:
try {
    URL pythonJar = PackagedProgramUtils.getPythonJar();
    if (pythonJar == null || !new File(pythonJar.toURI()).exists()) {
        System.err.println("flink-python JAR not found. Install it in opt/ or lib/.");
    }
} catch (Exception e) {
    System.err.println("Cannot locate flink-python JAR: " + e.getMessage());
}

When it happens

Trigger: Using -py or -pyfl options when the flink-python JAR is not present in FLINK_HOME/opt/ or on the classpath; the JAR exists but is corrupted or version-incompatible; PackagedProgramUtils.getPythonJar() cannot locate the Python jar in the expected directory.

Common situations: Minimal Flink installation without the Python component; version mismatch between the flink-python JAR and the core Flink distribution; custom container images that didn't include the opt/flink-python*.jar.

Related errors

• Bad syntax for classpath
• Can not register process function transformation translator.
• Cannot deserialize and unwrap accumulators properly.
• Cannot support file system for
• Class ' ' not found. Perhaps you forgot to add the module '…

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/1ce531e17d31e6b8. Report an issue: GitHub.

Appendix: source

Thrown at flink-clients/src/main/java/org/apache/flink/client/cli/ProgramOptionsUtils.java:92

                || line.hasOption(PYTHON_PATH.getOpt());
    }

    public static ProgramOptions createPythonProgramOptions(CommandLine line)
            throws CliArgsException {
        try {
            ClassLoader classLoader = getPythonClassLoader();
            Class<?> pythonProgramOptionsClazz =
                    Class.forName(
                            "org.apache.flink.client.cli.PythonProgramOptions", false, classLoader);
            Constructor<?> constructor =
                    pythonProgramOptionsClazz.getConstructor(CommandLine.class);
            return (ProgramOptions) constructor.newInstance(line);
        } catch (InstantiationException
                | InvocationTargetException
                | NoSuchMethodException
                | IllegalAccessException
                | ClassNotFoundException e) {
            throw new CliArgsException(
                    "Python command line option detected but the flink-python module seems to be missing "
                            + "or not working as expected.",
                    e);
        }
    }

    private static ClassLoader getPythonClassLoader() {
        try {
            return new URLClassLoader(
                    new URL[] {PackagedProgramUtils.getPythonJar()},
                    Thread.currentThread().getContextClassLoader());
        } catch (RuntimeException e) {
            LOG.warn(
                    "An attempt to load the flink-python jar from the \"opt\" directory failed, "
                            + "fall back to use the context class loader.",
                    e);
            return Thread.currentThread().getContextClassLoader();
        }

View on GitHub (pinned to 2f3c205e92)