apache/beam · error · Exception

At least " + session.countPendingErrors() + " error(s)…

Error message

At least " + session.countPendingErrors() + " error(s) occurred writing to Kudu

What it means

KuduServiceImpl.closeSession checks the Kudu session for pending errors after applying writes; if the KuduScanner/KuduSession accumulated row errors, the writer fails the whole operation with this Exception after logging a sample of the errors.

Solutions

Inspect the KuduTablet server / session pending errors logged ('Sample error: ...') to identify the failing rows
Fix the offending rows (validate schema, primary keys, nullability) before writing
Verify the target table exists and its schema matches the Rows being written
Catch this exception in the DoFn and route failed batches to a dead-letter sink

Example fix

// before
session.apply(insertWithWrongSchema);
// throws at closeSession: "At least 3 error(s) occurred writing to Kudu"
// after
if (!table.getSchema().equals(expectedSchema)) {
  throw new IllegalArgumentException("Kudu schema mismatch, refusing write");
}
session.apply(insertMatchingSchema);

Defensive patterns

Strategy: try-catch

Validate before calling

// before writing, verify table schema matches rows
if (!table.getSchema().equals(expectedSchema)) {
  throw new IllegalStateException("Kudu table schema drift detected");
}

Try / catch

try {
  kuduWriter.close(); // may throw Kudu write errors
} catch (Exception e) {
  if (e.getMessage().contains("error(s) occurred writing to Kudu")) {
    // inspect logged 'Sample error:' lines; route batch to dead-letter and retry
  }
}

Prevention

Validate rows against the Kudu schema (types, nullability, primary keys) before apply
Monitor Kudu tablet/table schema changes during long-running pipelines
Log full pending errors, not just the first three, in staging to catch data issues early

When it happens

Trigger: Writing rows to Kudu where the KuduSession reports countPendingErrors() > 0 at flush/close time — e.g., schema mismatches, constraint violations, non-existent tablets/table, or duplicate primary keys in the batch.

Common situations: Bulk inserts/updates into Kudu during Beam pipeline runs; bad data rows or a table whose schema changed while the pipeline runs; misconfigured masters causing write failures.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/07418e6e47adf25b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/kudu/src/main/java/org/apache/beam/sdk/io/kudu/KuduServiceImpl.java:116

    }

    @Override
    public void write(T entity) throws KuduException {
      checkState(session != null, "must call openSession() before writing");
      session.apply(formatFunction.apply(new TableAndRecord(table, entity)));
    }

    @Override
    public void closeSession() throws Exception {
      try {
        session.close();
        if (session.countPendingErrors() > 0) {
          LOG.error("At least {} errors occurred writing to Kudu", session.countPendingErrors());
          RowError[] errors = session.getPendingErrors().getRowErrors();
          for (int i = 0; errors != null && i < 3 && i < errors.length; i++) {
            LOG.error("Sample error: {}", errors[i]);
          }
          throw new Exception(
              "At least " + session.countPendingErrors() + " error(s) occurred writing to Kudu");
        }
      } finally {
        session = null;
      }
    }

    @Override
    public void close() throws Exception {
      client.close();
      client = null;
    }
  }

  /** Bounded reader of an Apache Kudu table. */
  class ReaderImpl extends BoundedSource.BoundedReader<T> {
    private final KuduIO.KuduSource<T> source;
    private KuduClient client;

View on GitHub (pinned to 12126d8942)