apache/beam · error · RuntimeException

Error when reading

Error message

Error when reading: %s

What it means

During Splunk sink setup, the SSL certificate file is fetched from GCS via FileSystems.matchSingleFileSpec and read fully into bytes. If any IOException occurs while matching/opening/reading the file, the code wraps it in a RuntimeException carrying the path. It usually means the path doesn't exist, isn't readable, or the GCS filesystem/credentials aren't configured.

Solutions

1. Verify the exact gs://bucket/path exists (gsutil ls) and the path is the file itself, not a directory or glob.
2. Grant the pipeline's service account roles/storage.objectViewer on the bucket.
3. Prefer uploading the cert via a supported scheme for your environment, or inline/pass the cert bytes directly instead of a GCS reference.
4. Check the wrapped cause (getCause()) to distinguish match failures from stream read failures.

Example fix

// before
SplunkIO.writeEvents("https://splunk:8088").withToken(token)
    .withCertFilePath("gs://mybucket/certs/splunk.crt")
// after (verify first)
gsutil ls gs://mybucket/certs/splunk.crt  # ensure it exists and is readable
SplunkIO.writeEvents("https://splunk:8088").withToken(token)
    .withCertFilePath("gs://mybucket/certs/splunk.crt")

// Java — verify the GCS object exists and is readable before configuring the sink
MatchResult r = FileSystems.match(Collections.singletonList("gs://bucket/certs/splunk.crt"));
if (r.status() != MatchResult.Status.OK || r.metadata().isEmpty()) {
  throw new IllegalStateException("Cert not found at gs://bucket/certs/splunk.crt");
}

try {
  byte[] cert = SplunkEventWriter.getCertFromGcsAsBytes(path);
} catch (RuntimeException e) {
  if (e.getCause() instanceof IOException) {
    throw new IllegalStateException("Unreadable cert path: " + path, e.getCause());
  }
  throw e;
}

When it happens

Trigger: Calling SplunkIO.writeSsl(...) or configuring the writer with a GCS cert path via withCertFilePath()/setup() where the path fails FileSystems.match, the object was deleted/renamed, or credentials lack storage.objects.get.

Common situations: Typos in gs:// paths, service accounts without GCS read permission, missing GCS/relay URL config in Dataflow, or referencing a local path when running on a worker where only GCS is mounted.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Error completing file copies with retries, sample: from
• Error trying to delete
• GcsSeekableByteChannels are read-only and cannot be…
• Interrupted backoff of file copies with retries, sample…
• Skipping dest existence is only supported within a bucket.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a383d67bfdcc1245. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/splunk/src/main/java/org/apache/beam/sdk/io/splunk/SplunkEventWriter.java:450

      receiver.output(error);
    }
  }

  /**
   * Reads a root CA certificate from GCS and returns it as raw bytes.
   *
   * @param filePath path to root CA cert in GCS
   * @return raw contents of cert
   * @throws RuntimeException thrown if not able to read or parse cert
   */
  public static byte[] getCertFromGcsAsBytes(String filePath) throws IOException {
    MatchResult.Metadata fileMetadata = FileSystems.matchSingleFileSpec(filePath);
    ReadableByteChannel channel = FileSystems.open(fileMetadata.resourceId());
    try (InputStream inputStream = Channels.newInputStream(channel)) {
      return IOUtils.toByteArray(inputStream);
    } catch (IOException e) {
      throw new RuntimeException("Error when reading: " + filePath, e);
    }
  }

  @VisibleForTesting
  static boolean isValidUrlFormat(String url) {
    Matcher matcher = URL_PATTERN.matcher(url);
    if (matcher.find()) {
      String host = matcher.group(2);
      return InetAddresses.isInetAddress(host) || InternetDomainName.isValid(host);
    }
    return false;
  }

  /**
   * Converts Nanoseconds to Milliseconds.
   *
   * @param ns time in nanoseconds
   * @return time in milliseconds

View on GitHub (pinned to 12126d8942)