{"record":{"id":"0c9e1ed78b6bd9aa","repo":"conductor-oss/conductor","slug":"unable-to-upload-payload-to-external-storage-for-w","errorCode":null,"errorMessage":"Unable to upload payload to external storage for workflow: %s","messagePattern":"Unable to upload payload to external storage for workflow: (.+?)","errorType":"exception","errorClass":"NonTransientException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/com/netflix/conductor/core/utils/ExternalPayloadStorageUtils.java","lineNumber":198,"sourceCode":"                    case WORKFLOW_OUTPUT:\n                        externalOutputPayloadStoragePath =\n                                uploadHelper(\n                                        payloadBytes, payloadSize, PayloadType.WORKFLOW_OUTPUT);\n                        ((WorkflowModel) entity)\n                                .externalizeOutput(externalOutputPayloadStoragePath);\n                        Monitors.recordExternalPayloadStorageUsage(\n                                ((WorkflowModel) entity).getWorkflowName(),\n                                ExternalPayloadStorage.Operation.WRITE.toString(),\n                                PayloadType.WORKFLOW_OUTPUT.toString());\n                        break;\n                }\n            }\n        } catch (TransientException | TerminateWorkflowException te) {\n            throw te;\n        } catch (Exception e) {\n            LOGGER.error(\n                    \"Unable to upload payload to external storage for workflow: {}\", workflowId, e);\n            throw new NonTransientException(\n                    \"Unable to upload payload to external storage for workflow: \" + workflowId, e);\n        }\n    }\n\n    @VisibleForTesting\n    String uploadHelper(\n            byte[] payloadBytes, long payloadSize, ExternalPayloadStorage.PayloadType payloadType) {\n        ExternalStorageLocation location =\n                externalPayloadStorage.getLocation(\n                        ExternalPayloadStorage.Operation.WRITE, payloadType, \"\", payloadBytes);\n        externalPayloadStorage.upload(\n                location.getPath(), new ByteArrayInputStream(payloadBytes), payloadSize);\n        return location.getPath();\n    }\n\n    @VisibleForTesting\n    void failTask(TaskModel task, PayloadType payloadType, String errorMsg) {\n        LOGGER.error(errorMsg);","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/conductor-oss/conductor/blob/cf7c3e4a8adfb158be778ab1ec525323c363cd3a/core/src/main/java/com/netflix/conductor/core/utils/ExternalPayloadStorageUtils.java#L180-L216","documentation":"Thrown by ExternalPayloadStorageUtils.verifyAndUpload when uploading a task or workflow payload to external storage fails for any non-transient reason — e.g. storage backend is unreachable with a non-transient error, the upload request is rejected, or serialization fails. TransientException and TerminateWorkflowException are re-thrown as-is; all other exceptions are wrapped in NonTransientException. This occurs when a payload exceeds the threshold size and needs to be offloaded to external storage.","triggerScenarios":"A task or workflow input/output payload exceeds the configured size threshold (triggering external storage upload), but the upload to the external storage backend fails — e.g. S3 returns a 403 (permission denied), 400 (malformed request), the configured bucket doesn't exist, or the ObjectMapper fails to serialize the payload.","commonSituations":"External storage bucket not created or misconfigured. IAM credentials lack write permissions. Bucket name or region misconfigured in application.properties. Payload contains non-serializable objects causing Jackson to fail during byte array conversion. Storage backend quota exceeded.","solutions":["Verify the external storage configuration is correct: bucket name, region, endpoint, and credentials in application.properties.","Ensure the storage credentials have write permissions (s3:PutObject or equivalent) on the target bucket/path.","Check the server log for the underlying cause exception to distinguish between a permissions error, a missing bucket, or a serialization failure.","If the payload contains non-serializable data, clean the task/workflow input or output before the upload threshold is hit.","Verify the external storage implementation bean is correctly wired (e.g. S3ExternalPayloadStorage, not the default no-op)."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Verify payload is serializable and within size limits before the upload path\ntry {\n    byte[] bytes = objectMapper.writeValueAsBytes(payload);\n    if (bytes.length > maxThresholdBytes) {\n        throw new IllegalStateException(\"Payload exceeds max threshold of \" + maxThresholdBytes);\n    }\n} catch (IOException e) {\n    throw new IllegalArgumentException(\"Payload cannot be serialized\", e);\n}","typeGuard":null,"tryCatchPattern":"try {\n    externalPayloadStorageUtils.verifyAndUpload(entity, payloadType);\n} catch (NonTransientException e) {\n    LOGGER.error(\"External storage upload failed for workflow — check storage config\", e);\n    throw e;\n} catch (TransientException te) {\n    // Retryable — the backend was temporarily unavailable\n    throw te;\n}","preventionTips":["Ensure external storage bucket exists and credentials have write permissions before deployment.","Test the external storage integration in a staging environment with representative payload sizes.","Monitor external storage write errors and alert on permission/quota issues.","Keep payload sizes reasonable — large payloads that hit external storage are a sign of design issues."],"tags":["external-storage","payload","upload","non-transient","configuration"],"backgroundTag":null,"analyzedSha":"cf7c3e4a8adfb158be778ab1ec525323c363cd3a","analyzedAt":"2026-08-14T03:33:19.897Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}