apache/beam · error · RuntimeException

Datadog write failed with status code

Error message

Datadog write failed with status code %d: %s

What it means

FailOnWriteErrorFn is the default failure handler of the Datadog SchemaTransform write. When a DatadogWriteError carries a non-null statusCode, processElement throws a RuntimeException embedding the HTTP status code and Datadog's status message, aborting the pipeline element.

Solutions

1. Verify the Datadog API key and app key are valid and have write permissions.
2. Check the status code in the message: 429/402 means rate/quota limits — add batching or reduce event rate.
3. Implement a custom failure handler (e.g. via a dead-letter output or retry sink) instead of the default FailOnWriteErrorFn.
4. Retry with backoff for transient 5xx status codes.

Example fix

// before
.write(); // default: throws on any Datadog error
// after
.apply("DatadogWrite", DatadogIO.write().withApi(...).withFailureSignal(...)); // route errors to a dead-letter PCollection

// Pre-flight: validate API key
HttpURLConnection c = (HttpURLConnection) new URL(datadogUrl + "/api/v1/validate").openConnection();
c.setRequestProperty("DD-API-KEY", apiKey);
if (c.getResponseCode() != 200) throw new IllegalStateException("Invalid Datadog API key");

static boolean isRetryable(Integer statusCode) {
  return statusCode != null && (statusCode >= 500 || statusCode == 429);
}

try {
  write.apply(datadogTransform);
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("Datadog write failed with status code 4")) {
    // fix credentials/payload; not retryable
  } else {
    // retry with backoff
  }
}

When it happens

Trigger: Datadog API responds with an HTTP error (e.g. 403 invalid API key, 429 rate limit, 413 payload too large) while writing events, and the default fail-on-error behavior is active.

Common situations: Expired or wrong Datadog API key, exceeding the intake events limit (402/429), or sending events larger than Datadog's size limit.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• Datadog write failed:
• Error trying to delete
• A function must be provided to convert the input type into…
• A PValue contained in
• A schema was provided without a data format (or viceversa)…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3e26d7b5beee8ecd. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/datadog/src/main/java/org/apache/beam/sdk/io/datadog/DatadogWriteSchemaTransformProvider.java:286

                  .build());
        } else {
          throw new RuntimeException(e);
        }
      }
    }
  }

  /**
   * A {@link DoFn} that throws a {@link RuntimeException} when a write error is encountered,
   * causing the pipeline to fail. This is the default error handling behavior when no error output
   * is configured.
   */
  static class FailOnWriteErrorFn extends DoFn<DatadogWriteError, Void> {
    @ProcessElement
    public void processElement(@Element DatadogWriteError error) {
      String message = error.statusMessage();
      if (error.statusCode() != null) {
        throw new RuntimeException(
            String.format(
                "Datadog write failed with status code %d: %s", error.statusCode(), message));
      } else {
        throw new RuntimeException("Datadog write failed: " + message);
      }
    }
  }
}

View on GitHub (pinned to 12126d8942)