apache/beam · error · GradleException

error occurred

Error message

error occurred

What it means

Beam's Python load-test Gradle tasks run the test module via a shell command with ignoreExitValue=true, then inspect the exit code in doLast. If the Python process exited non-zero, the build throws this generic GradleException('error occurred').

Solutions

1. Scroll up in the Gradle output to find the real Python/beam error printed before the generic exception.
2. Verify --test-pipeline-options JSON is correctly escaped (parseOptions only escapes double quotes).
3. Confirm the Python virtualenv (envdir) exists and has apache_beam installed; rerun installGcpTest.
4. Re-run the single load test directly with python -m <mainClass> to see the full traceback.

Example fix

// before
./gradlew :sdks:python:...:runLoadTest -PloadTestArgs='{"num_workers":1'
// after
./gradlew :sdks:python:...:runLoadTest -PloadTestArgs='{"num_workers": 1}'

try {
  exec("./gradlew runLoadTest -PloadTestArgs=...")
} catch (e) {
  log(e.output) // real Python failure is printed above the generic 'error occurred'
}

When it happens

Trigger: Running any Beam load test task (e.g. `./gradlew :sdks:python:apache_beam:testing:load_tests:runLoadTest`) where the invoked Python module fails — bad --test-pipeline-options, missing GCP credentials, or a crashed pipeline.

Common situations: Misquoted JSON in loadTestArgs pipeline options; the virtualenv at project.ext.envdir missing or broken; Dataflow job failure during a load test run on CI.

Related errors

• Unknown yamlTestSet: . Must be one of 'data', 'databases'…
• BigQueryMetastoreCatalog doesn't support Java 8
• Could not find the provided transforms config source
• Could not generate external transform wrappers due to error
• Could not generate protos due to error

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bff4d67748272f7f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/testing/load_tests/build.gradle:68

    }

    String requirementsTxtFileArg = project.findProperty(requirementsTxtFileProperty) ?: null
    if (requirementsTxtFileArg != null) {
    doFirst {
        exec {
          setWorkingDir "${project.rootDir}/sdks/python"
          executable 'sh'
          args '-c', "${project.ext.envdir}/bin/python -m pip install -r ${requirementsTxtFileArg}"
        }
     }
   }
    setWorkingDir "${project.rootDir}/sdks/python"
    commandLine 'sh', '-c', "${project.ext.envdir}/bin/python -m ${mainClass} --test-pipeline-options=\"${parseOptions(loadTestArgs)}\" --ignore-files \'.*py3\\d?\\.py\$\'"
    ignoreExitValue true

    doLast {
        if (executionResult.get().exitValue != 0) {
            throw new GradleException('error occurred')
        }
    }
}

def parseOptions(String option) {
    option.replace('\"', '\\"')
}

View on GitHub (pinned to 12126d8942)