prestodb/presto · error · PrestoException

NOT_SUPPORTED

NOT_SUPPORTED

Error message

Automatic writers scaling is not supported by Presto on Spark

What it means

createSparkRdd validates the fragment's partitioning handle before building the RDD. If the partitioning is SCALED_WRITER_DISTRIBUTION (automatic writer scaling), it throws PrestoException with code NOT_SUPPORTED because scaling writers dynamically is unsupported in Presto on Spark.

Source

Thrown at presto-spark-base/src/main/java/com/facebook/presto/spark/planner/PrestoSparkRddFactory.java:145

    public <T extends PrestoSparkTaskOutput> JavaPairRDD<MutablePartitionId, T> createSparkRdd(
            JavaSparkContext sparkContext,
            Session session,
            PlanFragment fragment,
            Map<PlanFragmentId, JavaPairRDD<MutablePartitionId, PrestoSparkMutableRow>> rddInputs,
            Map<PlanFragmentId, Broadcast<?>> broadcastInputs,
            PrestoSparkTaskExecutorFactoryProvider executorFactoryProvider,
            CollectionAccumulator<SerializedTaskInfo> taskInfoCollector,
            CollectionAccumulator<PrestoSparkShuffleStats> shuffleStatsCollector,
            TableWriteInfo tableWriteInfo,
            Class<T> outputType,
            TempStorage nativeTempStorage)
    {
        checkArgument(!fragment.getStageExecutionDescriptor().isStageGroupedExecution(), "unexpected grouped execution fragment: %s", fragment.getId());

        PartitioningHandle partitioning = fragment.getPartitioning();

        if (partitioning.equals(SCALED_WRITER_DISTRIBUTION)) {
            throw new PrestoException(NOT_SUPPORTED, "Automatic writers scaling is not supported by Presto on Spark");
        }

        checkArgument(!partitioning.equals(COORDINATOR_DISTRIBUTION), "COORDINATOR_DISTRIBUTION fragment must be run on the driver");
        checkArgument(!partitioning.equals(FIXED_BROADCAST_DISTRIBUTION), "FIXED_BROADCAST_DISTRIBUTION can only be set as an output partitioning scheme, and not as a fragment distribution");
        checkArgument(!partitioning.equals(FIXED_PASSTHROUGH_DISTRIBUTION), "FIXED_PASSTHROUGH_DISTRIBUTION can only be set as local exchange partitioning");

        // TODO: ARBITRARY_DISTRIBUTION is something very weird.
        // TODO: It doesn't have partitioning function, and it is never set as a fragment partitioning.
        // TODO: We should consider removing ARBITRARY_DISTRIBUTION.
        checkArgument(!partitioning.equals(ARBITRARY_DISTRIBUTION), "ARBITRARY_DISTRIBUTION is not expected to be set as a fragment distribution");

        if (partitioning.equals(SINGLE_DISTRIBUTION) ||
                partitioning.equals(FIXED_HASH_DISTRIBUTION) ||
                partitioning.equals(FIXED_ARBITRARY_DISTRIBUTION) ||
                partitioning.equals(SOURCE_DISTRIBUTION) ||
                partitioning.getConnectorId().isPresent()) {
            return createRdd(
                    sparkContext,

View on GitHub (pinned to 55bb57d202)

Solutions

Disable the scale_writers session property (SET SESSION scale_writers = false) for Spark deployments
Remove connector/table properties that request scaled writer distribution
Tune writer parallelism statically instead (e.g. repartition output) since dynamic scaling is unavailable

Example fix

// before
SET SESSION scale_writers = true;
// after
SET SESSION scale_writers = false; -- automatic writers scaling unsupported on Presto on Spark

Defensive patterns

Strategy: validation

Validate before calling

if ("true".equals(session.getSystemProperty("scale_writers"))) {
    throw new PrestoException(NOT_SUPPORTED, "scale_writers must be false on Presto on Spark");
}

Try / catch

try { rddFactory.createSparkRdd(...) } catch (PrestoException e) { if (e.getErrorCode().getCode() == NOT_SUPPORTED.toErrorCode().getCode()) { log.error("scaled writers unsupported", e); } throw e; }

Prevention

Never enable scale_writers in Spark deployment session catalogs
Use static repartitioning for write parallelism instead
Validate write-heavy query sessions in CI before production rollout

When it happens

Trigger: A query fragment's output partitioning resolves to SCALED_WRITER_DISTRIBUTION while the RDD is being created in createSparkRdd — typically when scale-writers session property is enabled for writing queries.

Common situations: Users enabling SET SESSION scale_writers = true (or scale_writers-supported connector settings) on Presto on Spark during heavy write queries; porting tuned write queries from Presto-on-Hive to Spark.

Understand the failure class

Background: Presto NOT_SUPPORTED error: what "not supported" means and how to fix it — this error's family across 3 libraries.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/7cbd59a612883eea. Report an issue: GitHub.