What causes "Unable to upload payload to external storage for workflow: %s" in conductor-oss/conductor?

Verify the external storage configuration is correct: bucket name, region, endpoint, and credentials in application.properties.

conductor-oss/conductor · error · NonTransientException

Unable to upload payload to external storage for workflow: %

Error message

Unable to upload payload to external storage for workflow: %s

What it means

Thrown by ExternalPayloadStorageUtils.verifyAndUpload when uploading a task or workflow payload to external storage fails for any non-transient reason — e.g. storage backend is unreachable with a non-transient error, the upload request is rejected, or serialization fails. TransientException and TerminateWorkflowException are re-thrown as-is; all other exceptions are wrapped in NonTransientException. This occurs when a payload exceeds the threshold size and needs to be offloaded to external storage.

Source

Thrown at core/src/main/java/com/netflix/conductor/core/utils/ExternalPayloadStorageUtils.java:198

                    case WORKFLOW_OUTPUT:
                        externalOutputPayloadStoragePath =
                                uploadHelper(
                                        payloadBytes, payloadSize, PayloadType.WORKFLOW_OUTPUT);
                        ((WorkflowModel) entity)
                                .externalizeOutput(externalOutputPayloadStoragePath);
                        Monitors.recordExternalPayloadStorageUsage(
                                ((WorkflowModel) entity).getWorkflowName(),
                                ExternalPayloadStorage.Operation.WRITE.toString(),
                                PayloadType.WORKFLOW_OUTPUT.toString());
                        break;
                }
            }
        } catch (TransientException | TerminateWorkflowException te) {
            throw te;
        } catch (Exception e) {
            LOGGER.error(
                    "Unable to upload payload to external storage for workflow: {}", workflowId, e);
            throw new NonTransientException(
                    "Unable to upload payload to external storage for workflow: " + workflowId, e);
        }
    }

    @VisibleForTesting
    String uploadHelper(
            byte[] payloadBytes, long payloadSize, ExternalPayloadStorage.PayloadType payloadType) {
        ExternalStorageLocation location =
                externalPayloadStorage.getLocation(
                        ExternalPayloadStorage.Operation.WRITE, payloadType, "", payloadBytes);
        externalPayloadStorage.upload(
                location.getPath(), new ByteArrayInputStream(payloadBytes), payloadSize);
        return location.getPath();
    }

    @VisibleForTesting
    void failTask(TaskModel task, PayloadType payloadType, String errorMsg) {
        LOGGER.error(errorMsg);

View on GitHub (pinned to cf7c3e4a8a)

Solutions

Verify the external storage configuration is correct: bucket name, region, endpoint, and credentials in application.properties.
Ensure the storage credentials have write permissions (s3:PutObject or equivalent) on the target bucket/path.
Check the server log for the underlying cause exception to distinguish between a permissions error, a missing bucket, or a serialization failure.
If the payload contains non-serializable data, clean the task/workflow input or output before the upload threshold is hit.
Verify the external storage implementation bean is correctly wired (e.g. S3ExternalPayloadStorage, not the default no-op).

Defensive patterns

Strategy: validation

Validate before calling

// Verify payload is serializable and within size limits before the upload path
try {
    byte[] bytes = objectMapper.writeValueAsBytes(payload);
    if (bytes.length > maxThresholdBytes) {
        throw new IllegalStateException("Payload exceeds max threshold of " + maxThresholdBytes);
    }
} catch (IOException e) {
    throw new IllegalArgumentException("Payload cannot be serialized", e);
}

Try / catch

try {
    externalPayloadStorageUtils.verifyAndUpload(entity, payloadType);
} catch (NonTransientException e) {
    LOGGER.error("External storage upload failed for workflow — check storage config", e);
    throw e;
} catch (TransientException te) {
    // Retryable — the backend was temporarily unavailable
    throw te;
}

Prevention

Ensure external storage bucket exists and credentials have write permissions before deployment.
Test the external storage integration in a staging environment with representative payload sizes.
Monitor external storage write errors and alert on permission/quota issues.
Keep payload sizes reasonable — large payloads that hit external storage are a sign of design issues.

When it happens

Trigger: A task or workflow input/output payload exceeds the configured size threshold (triggering external storage upload), but the upload to the external storage backend fails — e.g. S3 returns a 403 (permission denied), 400 (malformed request), the configured bucket doesn't exist, or the ObjectMapper fails to serialize the payload.

Common situations: External storage bucket not created or misconfigured. IAM credentials lack write permissions. Bucket name or region misconfigured in application.properties. Payload contains non-serializable objects causing Jackson to fail during byte array conversion. Storage backend quota exceeded.

Related errors

AI-assisted analysis of conductor-oss/conductor@cf7c3e4a8a (2026-08-14). Data as JSON: /api/errors/0c9e1ed78b6bd9aa. Report an issue: GitHub.