apache/beam · error · RuntimeException

Extract file timestamp failed: got file timestamp == 0.

Error message

Extract file timestamp failed: got file timestamp == 0.

What it means

FileIO's ExtractFilenameAndLastUpdateFn (used by FileIO.writeDynamic with withDestinationByFilenameAndTimestamp-style extraction) throws when MatchResult.Metadata.lastModifiedMillis() returns 0, meaning the filesystem did not provide a modification timestamp and a timestamp-based destination cannot be computed.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileIO.java:802

            .withWatermark(now);
      }
    }

    private static class ExtractFilenameFn
        implements SerializableFunction<MatchResult.Metadata, String> {
      @Override
      public String apply(MatchResult.Metadata input) {
        return input.resourceId().toString();
      }
    }

    private static class ExtractFilenameAndLastUpdateFn
        implements SerializableFunction<MatchResult.Metadata, KV<String, Long>> {
      @Override
      public KV<String, Long> apply(MatchResult.Metadata input) throws RuntimeException {
        long timestamp = input.lastModifiedMillis();
        if (0L == timestamp) {
          throw new RuntimeException("Extract file timestamp failed: got file timestamp == 0.");
        }
        return KV.of(input.resourceId().toString(), timestamp);
      }
    }
  }

  /** Implementation of {@link #readMatches}. */
  @AutoValue
  public abstract static class ReadMatches
      extends PTransform<PCollection<MatchResult.Metadata>, PCollection<ReadableFile>> {
    /** Enum to control how directories are handled. */
    public enum DirectoryTreatment {
      SKIP,
      PROHIBIT
    }

    abstract Compression getCompression();

View on GitHub (pinned to 12126d8942)

Solutions

Fix the upstream producer so files carry a valid mtime.
Use a different destination extraction that doesn't rely on lastModifiedMillis (custom SerializableFunction).
Check the storage connector/metadata provider returns real timestamps.

Example fix

// before
FileIO.writeDynamic().via(...).withDestination(extractFilenameAndLastUpdate())
// after
customFn: KV.of(input.resourceId().toString(), fallbackTimestampIfZero(input.lastModifiedMillis()))

Defensive patterns

Strategy: fallback

Validate before calling

long ts = metadata.lastModifiedMillis();
if (ts == 0L) { log.warn("Missing mtime for {}", metadata.resourceId()); }

Prevention

Verify your storage backend reports valid mtimes before using timestamp-based destinations.
Provide a fallback extraction function when mtime may be zero.
Avoid writing files to Beam inputs from systems that zero modification times.

When it happens

Trigger: Matching files whose metadata reports lastModifiedMillis()==0 — certain filesystems, objects created without mtime, or mocked/placeholder metadata — while applying a dynamic-destinations timestamp extraction function.

Common situations: Reading from filesystems that don't track modification times (some object stores, test fakes, or files written by systems that zero out mtime).

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f1702ce308e72383. Report an issue: GitHub.