apache/beam · error · RuntimeException

No such stream

Error message

No such stream: ${streamName}

What it means

FakeDatasetService is Beam's in-memory fake of the Google Cloud BigQuery Storage Write API used in tests. flush(streamName, offset) looks up the given stream name in the fake's internal writeStreams map; when the name is not present it throws RuntimeException("No such stream: ..."). This means the test harness was asked to flush a stream it never created (via createWriteStream) or has already removed/finalized.

Solutions

  1. Create the write stream before flushing: call createWriteStream for the stream name passed to flush.
  2. Verify the streamName string matches exactly the one returned by createWriteStream (no truncation or re-formatting).
  3. Ensure the test uses one FakeDatasetService instance consistently; construct a fresh fake (or clear its state) per test to avoid stale/absent streams.
  4. Check that the fake instance registered in the test's client options is the same one the writer is flushing against.

Example fix

// before
fakeDatasetService.flush("unknown-stream", 0);
// after
String streamName = fakeDatasetService.createWriteStream(tableUrn, WriteStreamRequest.newBuilder().setType(Type.BUFFERED).build()).getName();
fakeDatasetService.append(streamName, rows);
fakeDatasetService.flush(streamName, 0);
Defensive patterns

Strategy: try-catch

Validate before calling

boolean exists = fake.getWriteStreams().containsKey(streamName); // or track created names in a Set during the test
if (!exists) { throw new IllegalStateException("flush before createWriteStream: " + streamName); }

Try / catch

try {
  fake.flush(streamName, offset);
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("No such stream:")) {
    throw new AssertionError("Stream was not created on this FakeDatasetService: " + streamName, e);
  }
  throw e;
}

Prevention

When it happens

Trigger: Calling flush(streamName, offset) on the fake with a stream name that was never created with createWriteStream, a misspelled/interleaved stream name, or a stream from a previous test run's state that was not reset.

Common situations: Test code wiring a BigQueryIO write with the fake service but using stream names obtained from a different fake instance; reusing a fake across tests without clearing writeStreams; mixing real and fake client references in an integration test.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/47f2997570a97527. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/FakeDatasetService.java:930

      }

      @Override
      public void close() throws Exception {}

      @Override
      public void pin() {}

      @Override
      public void unpin() {}
    };
  }

  @Override
  public ApiFuture<FlushRowsResponse> flush(String streamName, long offset) {
    synchronized (FakeDatasetService.class) {
      Stream stream = writeStreams.get(streamName);
      if (stream == null) {
        throw new RuntimeException("No such stream: " + streamName);
      }
      stream.flush(offset);
    }
    return ApiFutures.immediateFuture(FlushRowsResponse.newBuilder().build());
  }

  @Override
  public ApiFuture<FinalizeWriteStreamResponse> finalizeWriteStream(String streamName) {
    synchronized (FakeDatasetService.class) {
      Stream stream = writeStreams.get(streamName);
      if (stream == null) {
        throw new RuntimeException("No such stream: " + streamName);
      }
      long numRows = stream.finalizeStream();
      return ApiFutures.immediateFuture(
          FinalizeWriteStreamResponse.newBuilder().setRowCount(numRows).build());
    }
  }

View on GitHub (pinned to 12126d8942)