prestodb/presto · error · PrestoException

HIVE_FILE_NOT_FOUND

HIVE_FILE_NOT_FOUND

Error message

Invalid path in Symlink manifest file %s: %s does not exist

What it means

Thrown when loading files via a SymlinkTextInputFormat manifest: a target path listed in the manifest does not match any HiveFileInfo gathered from the filesystem listing, so the referenced file does not exist. Presto refuses to produce splits for dangling manifest entries.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/HiveUtil.java:1385

                ((CachingDirectoryLister) directoryLister).invalidateDirectoryListCache(Optional.of(targetParent.toString()));

                targetParentHiveFileInfos.clear();
                targetParentHiveFileInfos.putAll(getTargetParentHiveFileInfoMap(
                        partition,
                        targetParent,
                        hiveDirectoryContext,
                        targetFilesystem,
                        directoryLister,
                        table,
                        namenodeStats));
            }
        }

        return currentTargetPaths.stream().map(targetPath -> {
            HiveFileInfo hiveFileInfo = targetParentHiveFileInfos.get(getPathWithoutSchemeAndAuthority(targetPath).toString());

            if (hiveFileInfo == null) {
                throw new PrestoException(HIVE_FILE_NOT_FOUND, String.format("Invalid path in Symlink manifest file %s: %s does not exist", path, targetPath));
            }

            return hiveFileInfo;
        }).collect(toImmutableList());
    }

    private static Map<String, HiveFileInfo> getTargetParentHiveFileInfoMap(
            Optional<Partition> partition,
            Path targetParent,
            HiveDirectoryContext hiveDirectoryContext,
            ExtendedFileSystem targetFilesystem,
            DirectoryLister directoryLister,
            Table table,
            NamenodeStats namenodeStats)
    {
        Map<String, HiveFileInfo> targetParentHiveFileInfos = new HashMap<>();
        Iterator<HiveFileInfo> hiveFileInfoIterator = directoryLister.list(targetFilesystem, table, targetParent, partition, namenodeStats, hiveDirectoryContext);

View on GitHub (pinned to 55bb57d202)

Solutions

1. Regenerate the symlink manifest so it only lists existing files
2. Restore the missing files at the referenced paths
3. Check scheme/authority normalization (use the same NameNode URI form as configured in the connector)
4. Remove stale manifests/partitions pointing at deleted data

Example fix

# before: manifest references deleted file
hdfs dfs -cat /t/symlink.txt
# /data/part-0001 (missing)
# after: rebuild manifest
hdfs dfs -ls /data | awk '{print $8}' > /t/symlink.txt

// before running a symlink-manifest query, verify each manifest line exists
for (String line : Files.readAllLines(manifestPath)) {
    Path p = new Path(line.trim());
    FileSystem fs = p.getFileSystem(conf);
    if (!fs.exists(p)) System.err.println("missing manifest target: " + p);
}

boolean manifestTargetExists(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.FileSystem fs)
        throws java.io.IOException {
    return fs.exists(p);
}

try { files = HiveUtil.getSymlinkFileInfos(...); }
catch (com.facebook.presto.spi.PrestoException e) {
    if ("HIVE_FILE_NOT_FOUND".equals(e.getErrorCode().getName())) { /* regenerate manifest or restore files */ }
    else throw e;
}

When it happens

Trigger: A symlink text input manifest (symlink.txt) lists file URIs that are absent from the actual filesystem listing — files deleted/moved after the manifest was written, typos in URIs, or scheme/authority mismatch such that the normalized path lookup misses.

Common situations: ETL overwrote/deleted files while a manifest still references them; manifest generated against a different cluster/namespace (HA name mismatch); retention jobs pruning data referenced by old manifests.

Related errors

• HIVE_FILESYSTEM_ERROR
• INVALID_SCHEMA_PROPERTY
• HIVE_FILESYSTEM_ERROR
• HIVE_FILE_NOT_FOUND
• HIVE_FILESYSTEM_ERROR

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/2433a185cc883460. Report an issue: GitHub.