apache/beam · error · IOException

Invalid state for Rewrite, from=

Error message

Invalid state for Rewrite, from=%s, to=%s, readyToEnqueue=%s

What it means

GcsUtilV1.Rewrite.enqueue throws this IOException when enqueue() is called on a Rewrite whose internal state flag readyToEnqueue is false. readyToEnqueue is only true after a successful initial rewrite response or a retryable error; it becomes false when the rewrite completes or the source is treated as missing. It is an internal state-machine guard in the GCS copy-with-rewrite helper.

Solutions

  1. Only call enqueue() when readyToEnqueue is true (check the public field first)
  2. Create a new Rewrite object instead of reusing a completed one
  3. Review the rewrite loop so each iteration uses the returned Rewrite result

Example fix

// before
rewrite.enqueue(batch); // may throw if already done
// after
if (rewrite.readyToEnqueue) {
  rewrite.enqueue(batch);
} else {
  LOG.info("Rewrite {} -> {} already finished", rewrite.from, rewrite.to);
}
Defensive patterns

Strategy: type-guard

Validate before calling

if (!rewrite.readyToEnqueue) { throw new IllegalStateException("Rewrite already done"); }

Type guard

boolean canEnqueue(GcsUtilV1.Rewrite r) { return r.readyToEnqueue; }

Try / catch

try {
  rewrite.enqueue(batch);
} catch (IOException e) {
  if (e.getMessage().contains("Invalid state for Rewrite")) {
    // create a fresh Rewrite or skip
  } else { throw e; }
}

Prevention

When it happens

Trigger: Enqueuing a Rewrite operation after it already completed, after it was marked done (readyToEnqueue=false), or calling enqueue twice on the same Rewrite object.

Common situations: Custom code reusing Rewrite objects across batches; driver bugs in iterative rewrite loops that don't re-check readyToEnqueue after each response.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8531a06818326eb5. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilV1.java:1044

    public @Nullable GoogleJsonError getLastError() {
      return lastError;
    }

    public GcsPath getFrom() {
      return from;
    }

    public GcsPath getTo() {
      return to;
    }

    public boolean isMetadataOperation() {
      return performDelete || from.getBucket().equals(to.getBucket());
    }

    public void enqueue(BatchInterface batch) throws IOException {
      if (!readyToEnqueue) {
        throw new IOException(
            String.format(
                "Invalid state for Rewrite, from=%s, to=%s, readyToEnqueue=%s",
                from, to, readyToEnqueue));
      }
      if (!performDelete) {
        batch.queue(rewriteRequest, this);
        return;
      }
      Storage.Objects.Delete deleteRequest =
          storageClient.objects().delete(from.getBucket(), from.getObject());
      batch.queue(
          deleteRequest,
          new JsonBatchCallback<Void>() {
            @Override
            public void onSuccess(Void obj, HttpHeaders responseHeaders) {
              LOG.debug("Successfully deleted {} after moving to {}", from, to);
              readyToEnqueue = false;
              lastError = null;

View on GitHub (pinned to 12126d8942)