apache/beam · error · RuntimeError

Request to %s failed with status %d: %s

Error message

Request to %s failed with status %d: %s

What it means

FlinkRestServer.request wraps all Flink master REST calls (get/post/delete). When the response status differs from expected (default 200), it raises RuntimeError with the URL, actual status code, and response body — the Flink cluster rejected or failed to serve the request.

Source

Thrown at sdks/python/apache_beam/runners/portability/flink_uber_jar_job_server.py:134

      job_id,
      job_name,
      pipeline,
      options,
      artifact_port=0):
    super().__init__(
        executable_jar,
        job_id,
        job_name,
        pipeline,
        options,
        artifact_port=artifact_port)
    self._master_url = master_url

  def request(self, method, path, expected_status=200, **kwargs):
    url = '%s/%s' % (self._master_url, path)
    response = method(url, **kwargs)
    if response.status_code != expected_status:
      raise RuntimeError(
          "Request to %s failed with status %d: %s" %
          (url, response.status_code, response.text))
    if response.text:
      return response.json()

  def get(self, path, **kwargs):
    return self.request(requests.get, path, **kwargs)

  def post(self, path, **kwargs):
    return self.request(requests.post, path, **kwargs)

  def delete(self, path, **kwargs):
    return self.request(requests.delete, path, **kwargs)

  def run(self):
    self._stop_artifact_service()

    # Upload the jar and start the job.

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the status code and body in the RuntimeError to identify the cause (404 = not found, 403 = denied, 500 = cluster error).
2. Verify master_url / --flink_master points at the correct, reachable JobManager REST endpoint.
3. Confirm the requested job_id/jar id still exists (GET /jobs or /jars) before mutating calls.
4. Retry on transient 5xx or restart the Flink cluster if errors persist.

Example fix

// before
client.post('jars/upload', files=...)  # RuntimeError status 403
// after
# fix flink-conf REST/auth settings, then:
resp = client.post('jars/upload', files=...)
assert resp['status'] == 'success'

import requests
ok = requests.get(f'{master_url}/overview', timeout=5).status_code == 200
assert ok, f"Flink master unreachable: {master_url}"

from tenacity import retry, stop_after_attempt, wait_exponential

@retry(stop=stop_after_attempt(3), wait=wait_exponential())
def safe_get(client, path):
    try:
        return client.get(path)
    except RuntimeError as e:
        if 'status 404' in str(e):
            raise  # not retryable: entity missing
        raise  # or retry transient 5xx per policy

When it happens

Trigger: Any REST call through get/post/delete returning a non-expected status: uploading jars when upload is restricted, deleting/getting an unknown job or jar id, cluster error states, or proxies returning 403/404/500 pages.

Common situations: Wrong --flink_master endpoint; job/jar already terminated or cleaned up; Flink REST API disabled or version mismatch; reverse proxy/auth intercepting requests.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• Timeout {timeout} exceeded while completing request: {reques
• could not complete request
• Unable to parse jar URL "%s". If using a full URL, make sure
• Unable to parse jar URL "%s". If using a full URL, make sure
• Job {} does not exist

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fc908181a294b297. Report an issue: GitHub.