apache/flink · error · RuntimeException

Cannot find the partition value from path for partition: %s

Error message

Cannot find the partition value from path for partition: %s

What it means

FileInfoExtractorBulkFormat reads partition column values from the file path using PartitionPathUtils.extractPartitionSpecFromPath. For each declared partition column, it looks up the field name in the extracted partition spec map. If the field name is not found (the path does not contain a matching partition directory), it throws a RuntimeException.

Source

Thrown at flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileInfoExtractorBulkFormat.java:147

        // Fill the metadata + partition columns row
        final GenericRowData fileInfoRowData =
                new GenericRowData(metadataColumnsFunctions.size() + partitionColumnTypes.size());
        int fileInfoRowIndex = 0;
        for (; fileInfoRowIndex < metadataColumnsFunctions.size(); fileInfoRowIndex++) {
            fileInfoRowData.setField(
                    fileInfoRowIndex,
                    metadataColumnsFunctions.get(fileInfoRowIndex).getValue(split));
        }
        if (!partitionColumnTypes.isEmpty()) {
            final LinkedHashMap<String, String> partitionSpec =
                    PartitionPathUtils.extractPartitionSpecFromPath(split.path());
            for (int partitionFieldIndex = 0;
                    fileInfoRowIndex < fileInfoRowData.getArity();
                    fileInfoRowIndex++, partitionFieldIndex++) {
                final String fieldName = partitionColumnTypes.get(partitionFieldIndex).getKey();
                final DataType fieldType = partitionColumnTypes.get(partitionFieldIndex).getValue();
                if (!partitionSpec.containsKey(fieldName)) {
                    throw new RuntimeException(
                            "Cannot find the partition value from path for partition: "
                                    + fieldName);
                }

                String valueStr = partitionSpec.get(fieldName);
                valueStr = valueStr.equals(defaultPartName) ? null : valueStr;
                fileInfoRowData.setField(
                        fileInfoRowIndex,
                        PartitionPathUtils.convertStringToInternalValue(valueStr, fieldType));
            }
        }

        // This row is going to be reused for every record
        final EnrichedRowData producedRowData =
                new EnrichedRowData(fileInfoRowData, this.extendedRowIndexMapping);

        return RecordMapperWrapperRecordIterator.wrapReader(
                superReader,

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Verify the file path matches the partition directory structure expected by the table DDL. Ensure directories follow the pattern key=value/.
2. Check the table DDL partition columns match the directory names in the file system path.
3. If the data is genuinely unpartitioned, remove the partition key declarations from the table definition.

Example fix

-- before
CREATE TABLE t (a INT, b STRING) PARTITIONED BY (year, month)
-- files at: /data/file.parquet  (no year= / month= dirs)

-- after
-- ensure files are at: /data/year=2024/month=01/file.parquet
-- OR remove partitioning if data is not partitioned:
CREATE TABLE t (a INT, b STRING) WITH ('path'='file:///data', 'format'='parquet')

// Validate partition paths before reading
for (String partitionKey : partitionKeys) {
    String partitionDir = extractPartitionFromPath(filePath, partitionKey);
    if (partitionDir == null) {
        throw new RuntimeException(
            "Partition column '" + partitionKey + "' not found in path: " + filePath);
    }
}

When it happens

Trigger: The table declares partition columns (partitionColumnTypes is non-empty) but the actual file path does not contain the expected partition directory structure. For example, the table expects partitions like /path/year=2024/month=01/ but the file is at /path/data.parquet without partition directories.

Common situations: Mismatch between the table's partition key definitions and the actual directory structure of the data files. Files placed outside the expected partition directory hierarchy. Table DDL changed to add partition keys but data files not moved to the correct partition paths. Using __HIVE_DEFAULT_PARTITION__ incorrectly.

Related errors

• The given offset is not contained in the any block.
• totalValueCount == 0
• Failed to create enumerator for sourceIndex={currentSourceIn
• Invalid version %d
• Invalid option %s. Must be a positive integer.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/6fb8548639595c14. Report an issue: GitHub.