prestodb/presto · error · IllegalArgumentException

Unexpected fragment partitioning %s, fragmentId: %s

Error message

Unexpected fragment partitioning %s, fragmentId: %s

What it means

After checking each known partitioning scheme, createSparkRdd falls through to a default branch and throws IllegalArgumentException for any fragment partitioning handle it does not recognize, including the fragment ID in the message. This indicates a plan partitioning that the Spark RDD factory cannot map to an RDD creation strategy.

Source

Thrown at presto-spark-base/src/main/java/com/facebook/presto/spark/planner/PrestoSparkRddFactory.java:176

                partitioning.equals(FIXED_HASH_DISTRIBUTION) ||
                partitioning.equals(FIXED_ARBITRARY_DISTRIBUTION) ||
                partitioning.equals(SOURCE_DISTRIBUTION) ||
                partitioning.getConnectorId().isPresent()) {
            return createRdd(
                    sparkContext,
                    session,
                    fragment,
                    executorFactoryProvider,
                    taskInfoCollector,
                    shuffleStatsCollector,
                    tableWriteInfo,
                    rddInputs,
                    broadcastInputs,
                    outputType,
                    nativeTempStorage);
        }
        else {
            throw new IllegalArgumentException(format("Unexpected fragment partitioning %s, fragmentId: %s", partitioning, fragment.getId()));
        }
    }

    private <T extends PrestoSparkTaskOutput> JavaPairRDD<MutablePartitionId, T> createRdd(
            JavaSparkContext sparkContext,
            Session session,
            PlanFragment fragment,
            PrestoSparkTaskExecutorFactoryProvider executorFactoryProvider,
            CollectionAccumulator<SerializedTaskInfo> taskInfoCollector,
            CollectionAccumulator<PrestoSparkShuffleStats> shuffleStatsCollector,
            TableWriteInfo tableWriteInfo,
            Map<PlanFragmentId, JavaPairRDD<MutablePartitionId, PrestoSparkMutableRow>> rddInputs,
            Map<PlanFragmentId, Broadcast<?>> broadcastInputs,
            Class<T> outputType,
            TempStorage nativeTempStorage)
    {
        checkInputs(fragment.getRemoteSourceNodes(), rddInputs, broadcastInputs);

View on GitHub (pinned to 55bb57d202)

Solutions

1. Align Presto versions across coordinator and workers to avoid unknown partitioning handles from newer plans
2. Log/examine the partitioning handle in the message and identify which connector or feature produced it
3. Re-rewrite the query to avoid connector-specific distribution requirements (e.g. disable connector bucketed/scaled distribution features)
4. File/patch support for the missing partitioning in PrestoSparkRddFactory's if-else chain

Example fix

// before
throw new IllegalArgumentException(format("Unexpected fragment partitioning %s, fragmentId: %s", partitioning, fragment.getId()));
// after (add explicit handling earlier)
if (partitioning.equals(SOME_NEW_DISTRIBUTION)) {
    return createDefaultRdd(partitioning, fragment, ...);
}

Set<PartitioningHandle> supported = ImmutableSet.of(SYSTEM_DISTRIBUTION, SINGLE_DYNAMIC_DISTRIBUTION, FIXED_HASH_DISTRIBUTION, ...);
checkArgument(supported.contains(fragment.getPartitioning()), "unsupported partitioning: " + fragment.getPartitioning());

boolean isSupportedPartitioning(PartitioningHandle p) { return p != null && SparkSupportedPartitionings.ALL.contains(p); }

try { rdd = rddFactory.createSparkRdd(...); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Unexpected fragment partitioning")) { log.error("unsupported partitioning " + e.getMessage(), e); } throw e; }

When it happens

Trigger: createSparkRdd receives a fragment whose getPartitioning() is not one of the handled schemes (system distributions, single-dynamic, fixed hash/passthrough/broadcast, scaled writer, coordinator — depending on the if-else chain).

Common situations: Coordinator/executor version skew so newer partitioning handles reach the factory; connector-specific partitioning handles not understood by Spark; internal planner changes introducing a new distribution not yet mapped in the RDD factory.

Related errors

• non-native (java) execution requires only one scheduledSplit
• Invalid day-time interval:
• Invalid year-month interval:
• dictionarySourceIds must be the same
• blocks is empty

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/e18e3ca257993d8e. Report an issue: GitHub.