{"record":{"id":"f1702ce308e72383","repo":"apache/beam","slug":"extract-file-timestamp-failed-got-file-timestamp-0","errorCode":null,"errorMessage":"Extract file timestamp failed: got file timestamp == 0.","messagePattern":"Extract file timestamp failed: got file timestamp == 0\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileIO.java","lineNumber":802,"sourceCode":"            .withWatermark(now);\n      }\n    }\n\n    private static class ExtractFilenameFn\n        implements SerializableFunction<MatchResult.Metadata, String> {\n      @Override\n      public String apply(MatchResult.Metadata input) {\n        return input.resourceId().toString();\n      }\n    }\n\n    private static class ExtractFilenameAndLastUpdateFn\n        implements SerializableFunction<MatchResult.Metadata, KV<String, Long>> {\n      @Override\n      public KV<String, Long> apply(MatchResult.Metadata input) throws RuntimeException {\n        long timestamp = input.lastModifiedMillis();\n        if (0L == timestamp) {\n          throw new RuntimeException(\"Extract file timestamp failed: got file timestamp == 0.\");\n        }\n        return KV.of(input.resourceId().toString(), timestamp);\n      }\n    }\n  }\n\n  /** Implementation of {@link #readMatches}. */\n  @AutoValue\n  public abstract static class ReadMatches\n      extends PTransform<PCollection<MatchResult.Metadata>, PCollection<ReadableFile>> {\n    /** Enum to control how directories are handled. */\n    public enum DirectoryTreatment {\n      SKIP,\n      PROHIBIT\n    }\n\n    abstract Compression getCompression();\n","sourceCodeStart":784,"sourceCodeEnd":820,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileIO.java#L784-L820","documentation":"FileIO's ExtractFilenameAndLastUpdateFn (used by FileIO.writeDynamic with withDestinationByFilenameAndTimestamp-style extraction) throws when MatchResult.Metadata.lastModifiedMillis() returns 0, meaning the filesystem did not provide a modification timestamp and a timestamp-based destination cannot be computed.","triggerScenarios":"Matching files whose metadata reports lastModifiedMillis()==0 — certain filesystems, objects created without mtime, or mocked/placeholder metadata — while applying a dynamic-destinations timestamp extraction function.","commonSituations":"Reading from filesystems that don't track modification times (some object stores, test fakes, or files written by systems that zero out mtime).","solutions":["Fix the upstream producer so files carry a valid mtime.","Use a different destination extraction that doesn't rely on lastModifiedMillis (custom SerializableFunction).","Check the storage connector/metadata provider returns real timestamps."],"exampleFix":"// before\nFileIO.writeDynamic().via(...).withDestination(extractFilenameAndLastUpdate())\n// after\ncustomFn: KV.of(input.resourceId().toString(), fallbackTimestampIfZero(input.lastModifiedMillis()))","handlingStrategy":"fallback","validationCode":"long ts = metadata.lastModifiedMillis();\nif (ts == 0L) { log.warn(\"Missing mtime for {}\", metadata.resourceId()); }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Verify your storage backend reports valid mtimes before using timestamp-based destinations.","Provide a fallback extraction function when mtime may be zero.","Avoid writing files to Beam inputs from systems that zero modification times."],"tags":["java","apache-beam","fileio","timestamp"],"backgroundTag":"invalid-config-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}