apache/beam · error · GradleException

-PuseWheelDistribution is set for the task but the host…

Error message

-PuseWheelDistribution is set for the task but the host system platform is not compatible with Dataflow worker container image.

What it means

Beam's Dataflow Python test-suite Gradle config validates that the host CPU architecture is amd64 when -PuseWheelDistribution is requested. The prebuilt Linux wheel matches the Dataflow worker container (amd64); running on other architectures (e.g. arm64 Apple Silicon) would produce a wheel/image mismatch, so the build fails fast.

Solutions

1. Run the build on an x86_64/amd64 host or an amd64 container (e.g. docker run --platform linux/amd64).
2. Drop -PuseWheelDistribution so the task installs from the sdist instead of the prebuilt wheel.
3. Use Rosetta/AMD64 emulation for local development on Apple Silicon.

Example fix

// before
./gradlew :sdks:python:test-suites:dataflow:pyxxx:installGcpTest -PuseWheelDistribution
// after (arm64 host)
./gradlew :sdks:python:test-suites:dataflow:pyxxx:installGcpTest  # no wheel flag

import platform
if platform.machine() not in ('x86_64', 'amd64'):
    raise SystemExit('Drop -PuseWheelDistribution or run on amd64')

When it happens

Trigger: Running any Dataflow Python test task with -PuseWheelDistribution on a machine whose os.arch is not amd64 (e.g. aarch64/arm64 Macs or ARM CI runners).

Common situations: Developers on Apple Silicon or ARM Graviton CI nodes building Dataflow test jobs that rely on the prebuilt wheel distribution.

Understand the failure class

Background: "unsupported platform" / "not supported on this platform" errors: what they mean and how to fix them — this error's family across 47 libraries.

Related errors

• Can not query metrics. Job id is unknown.
• Coder for the GroupByKey operation
• CombineFn.setup and CombineFn.teardown are not supported…
• Could not find element
• Could not translate the internal step name %r.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/cb71bf3adbf13695. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/test-suites/dataflow/common.gradle:44

        : ''

// Basic test options for ITs running on Jenkins.
def basicTestOpts = [
    "--capture=no",  // print stdout instantly
    "--numprocesses=8",  // run tests in parallel
    "--timeout=4500", // timeout of whole command execution
    "--color=yes", // console color
    "--log-cli-level=INFO" //log level info
]

dependencies {
  distTarBall project(path: ":sdks:python", configuration: "distTarBall")
}

task initializeForDataflowJob{
  def wheelCompatible = "amd64".equalsIgnoreCase(System.getProperty("os.arch"))
  if (!wheelCompatible && project.hasProperty('useWheelDistribution')) {
      throw new GradleException('-PuseWheelDistribution is set for the task but the ' +
      'host system platform is not compatible with Dataflow worker container image.')
  }
  dependsOn 'installGcpTest'

  if (project.hasProperty('useWheelDistribution')) {
    dependsOn ":sdks:python:bdistPy${pythonVersionNumber}linux"

    doLast {
      def cibwArchs = project.rootProject.findProperty('cibwArchs')
      def wheelInclude = cibwArchs == 'aarch64' ?
          "**/apache_beam-*cp${pythonVersionNumber}*aarch64*.whl" :
          "**/apache_beam-*cp${pythonVersionNumber}*manylinux*.whl"
      def collection = project.fileTree(project.project(':sdks:python').buildDir){
          include wheelInclude
      }
      // sdkLocation ext is set at execution time
      String packageFilename = collection.singleFile.toString()
      project.ext.sdkLocation = packageFilename

View on GitHub (pinned to 12126d8942)