apache/iceberg · error · NotFoundException

File %s is referenced in the logs of Delta Lake table at %s,

Error message

File %s is referenced in the logs of Delta Lake table at %s, but cannot be found in the storage

What it means

Thrown when a Delta log action references a data file (parquet) that no longer exists in the table's storage. The action builds an InputFile for the full path and checks existence before reading metrics; a missing file makes the migration abort since the Iceberg snapshot would reference a dangling file.

Source

Thrown at delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeTableAction.java:369

      RemoveFile removeFile = (RemoveFile) action;
      path = removeFile.getPath();
      nullableFileSize = removeFile.getSize().orElse(null);
      partitionValues = removeFile.getPartitionValues();
    } else {
      throw new ValidationException(
          "Unexpected action type for Delta Lake: %s", action.getClass().getSimpleName());
    }

    String fullFilePath = getFullFilePath(path, deltaLog.getPath().toString());
    // For unpartitioned table, the partitionValues should be an empty map rather than null
    Preconditions.checkArgument(
        partitionValues != null,
        String.format("File %s does not specify a partitionValues", fullFilePath));

    FileFormat format = determineFileFormatFromPath(fullFilePath);
    InputFile file = deltaLakeFileIO.newInputFile(fullFilePath);
    if (!file.exists()) {
      throw new NotFoundException(
          "File %s is referenced in the logs of Delta Lake table at %s, but cannot be found in the storage",
          fullFilePath, deltaTableLocation);
    }

    // If the file size is not specified, the size should be read from the file
    if (nullableFileSize != null) {
      fileSize = nullableFileSize;
    } else {
      fileSize = file.getLength();
    }

    // get metrics from the file
    MetricsConfig metricsConfig = MetricsConfig.forTable(table);
    String nameMappingString = table.properties().get(TableProperties.DEFAULT_NAME_MAPPING);
    NameMapping nameMapping =
        nameMappingString != null ? NameMappingParser.fromJson(nameMappingString) : null;
    Metrics metrics = getMetricsForFile(file, format, metricsConfig, nameMapping);

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Verify the file truly is absent at fullFilePath in the object store; fix the table location/IO prefix if the path is just wrong.
2. Restore the missing files (versioning/undelete on S3, backup) or restore the table to a snapshot whose files exist.
3. Run Delta Lake VACUUM with correct retention on a healthy copy and snapshot a version where all referenced files exist; never delete log-referenced files manually.
4. If files are intentionally gone, rewrite the Delta log (e.g. checkpoint/REORG or re-create table) so the log only references existing files, then re-run the migration.

Example fix

// before: snapshot points at a moved table location
DeltaLakeCatalogOptions.WAREHOUSE_LOCATION, "s3://old-bucket/table"
// after: point the action at the location where data files actually live
DeltaLakeCatalogOptions.WAREHOUSE_LOCATION, "s3://current-bucket/table"

InputFile f = fileIO.newInputFile(fullPath);
if (!f.exists()) { throw new IllegalStateException("Missing file before snapshot: " + fullPath); }

java.util.function.Predicate<String> fileExists = p -> fileIO.newInputFile(p).exists();

try {
  snapshotTable.execute();
} catch (NotFoundException e) {
  // log fullFilePath from message, verify storage location/restore file
}

When it happens

Trigger: Running SnapshotDeltaLakeTable on a Delta table whose _delta_log still contains AddFile entries for files that were deleted out-of-band (manual deletion, lifecycle policies, vacuum removing active files, wrong table location configured so paths resolve outside the real data).

Common situations: Deleting parquet files from S3/HDFS/ADLS manually or via bucket lifecycle rules without running Delta VACUUM; misconfigured deltaTableLocation/deltaLog path prefix so getFullFilePath points to a wrong prefix; copying _delta_log without its data files to a new location.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• RuntimeIOException
• Failed to read manifest file: %s
• Failed to write manifest
• Delta Lake table at %s contains no constructable snapshot
• The action %s's is unsupported

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/3f071e2ce638d0c4. Report an issue: GitHub.