apache/beam · error · RuntimeException

Error during Gemini inference request

Error message

Error during Gemini inference request

What it means

Any exception thrown while executing the Gemini inference call (network errors, API errors, deserialization problems) is caught in GeminiModelHandler.request() and rethrown as a RuntimeException with the message "Error during Gemini inference request", preserving the original as the cause. It is a generic wrapper so callers get a uniform failure type from the handler.

Solutions

1. Inspect the cause (e.getCause()) to identify the real failure.
2. Verify API key / Vertex AI credentials and network reachability to the endpoint.
3. Check the model name and request size against current Gemini API limits and quotas.
4. Pin/upgrade the Google GenAI SDK version to one compatible with your response parsing.

Example fix

// before
} catch (Exception e) { LOG.error("inference failed"); }
// after
try { results = handler.request(batch); }
catch (RuntimeException e) { LOG.error("Gemini inference failed", e.getCause()); throw e; }

if (apiKeyOrCredentialsMissing()) throw new IllegalStateException("configure Gemini credentials before inference");

try { results = handler.request(batch); } catch (RuntimeException e) { Throwable cause = e.getCause(); if (cause instanceof java.net.ConnectException) { /* backoff/retry */ } else if (cause instanceof com.google.genai.ApiException apiEx && apiEx.code() == 429) { /* rate-limit backoff */ } else { throw e; } }

When it happens

Trigger: modelHandler.request(inputs) invoked when the underlying GeminiRequestFunction throws: connectivity failures, 4xx/5xx from the Gemini or Vertex AI endpoint, invalid model name, auth failures, or response parsing errors.

Common situations: Expired or missing credentials, wrong model name, quota exhaustion (429), transient network outages, and SDK response-shape changes after upgrading the Google GenAI client library.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• Cannot infer schema with a circular reference. Class
• Failed to build transform
• Failed to parse secret option
• 2xx codes should not be exceptions. Got status code
• A 'datagen' table requires either 'rows-per-second' (for…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b501d37afa7bd51a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/ml/inference/gemini/src/main/java/org/apache/beam/sdk/ml/inference/gemini/GeminiModelHandler.java:83

  }

  @Override
  public Iterable<PredictionResult<InputT, OutputT>> request(List<InputT> input) {
    try {
      GeminiRequestFunction<InputT, OutputT> requestFn = modelParameters.getRequestFn();
      List<OutputT> responses = requestFn.apply(modelParameters.getModelName(), input, client);

      if (responses.size() != input.size()) {
        throw new IllegalStateException("Number of responses must match number of inputs");
      }

      List<PredictionResult<InputT, OutputT>> results = new ArrayList<>();
      for (int i = 0; i < input.size(); i++) {
        results.add(PredictionResult.create(input.get(i), responses.get(i)));
      }
      return results;
    } catch (Exception e) {
      throw new RuntimeException("Error during Gemini inference request", e);
    }
  }
}

View on GitHub (pinned to 12126d8942)