{"record":{"id":"cd618a17df25a0a0","repo":"conductor-oss/conductor","slug":"error-communicating-with-s3-operation-s-payloa","errorCode":null,"errorMessage":"Error communicating with S3 - operation:%s, payloadType: %s, path: %s","messagePattern":"Error communicating with S3 - operation:(.+?), payloadType: (.+?), path: (.+?)","errorType":"exception","errorClass":"TransientException","httpStatus":null,"severity":"error","filePath":"awss3-storage/src/main/java/com/netflix/conductor/s3/storage/S3PayloadStorage.java","lineNumber":133,"sourceCode":"\n                GetObjectPresignRequest presignRequest =\n                        GetObjectPresignRequest.builder()\n                                .signatureDuration(signatureDuration)\n                                .getObjectRequest(getObjectRequest)\n                                .build();\n\n                presignedUrl = s3Presigner.presignGetObject(presignRequest).url().toString();\n            }\n\n            externalStorageLocation.setUri(presignedUrl);\n            return externalStorageLocation;\n        } catch (SdkException e) {\n            String msg =\n                    String.format(\n                            \"Error communicating with S3 - operation:%s, payloadType: %s, path: %s\",\n                            operation, payloadType, path);\n            LOGGER.error(msg, e);\n            throw new TransientException(msg, e);\n        } catch (Exception e) {\n            String msg = \"Error generating presigned URL\";\n            LOGGER.error(msg, e);\n            throw new NonTransientException(msg, e);\n        }\n    }\n\n    /**\n     * Uploads the payload to the given s3 object key. It is expected that the caller retrieves the\n     * object key using {@link #getLocation(Operation, PayloadType, String)} before making this\n     * call.\n     *\n     * @param path the s3 key of the object to be uploaded\n     * @param payload an {@link InputStream} containing the json payload which is to be uploaded\n     * @param payloadSize the size of the json payload in bytes\n     */\n    @Override\n    public void upload(String path, InputStream payload, long payloadSize) {","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/conductor-oss/conductor/blob/cf7c3e4a8adfb158be778ab1ec525323c363cd3a/awss3-storage/src/main/java/com/netflix/conductor/s3/storage/S3PayloadStorage.java#L115-L151","documentation":"Thrown as TransientException (wrapping an SdkException) by S3PayloadStorage.getLocation when an AWS SDK call during presigned-URL generation fails with an SdkException. SdkException covers transport/credential/throttling/5xx-class failures that are typically retried by the SDK but can still propagate; conductor marks them Transient so callers can retry. The message embeds operation (READ/WRITE), payloadType, and path for diagnostics.","triggerScenarios":"Network error reaching S3, expired/invalid AWS credentials, IAM denial on the bucket/object, S3 throttling (SlowDown), or a 5xx from S3 during presignPutObject/presignGetObject. Caught in the SdkException handler of getLocation.","commonSituations":"IAM role without s3:GetObject/s3:PutObject on the payload bucket; transient network blip; S3 regional outage/throttling; STS credentials expired before the presign call; misconfigured region/endpoint.","solutions":["Retry the operation — TransientException signals a retriable failure; use exponential backoff.","Verify the conductor instance's AWS credentials/role have s3:GetObject and s3:PutObject on the configured bucket.","Check the region/endpoint configuration matches the bucket."],"exampleFix":"// before — single attempt that propagates TransientException\nString uri = storage.getLocation(Operation.WRITE, PayloadType.WORKFLOW_INPUT, key).getUri();\n// after — retry with backoff for TransientException\nRetryUtils.retryOn(TransientException.class, 5, Duration.ofMillis(200),\n    () -> storage.getLocation(Operation.WRITE, PayloadType.WORKFLOW_INPUT, key).getUri());","handlingStrategy":"retry","validationCode":"import com.netflix.conductor.common.run.ExternalStorageLocation;\nimport software.amazon.awssdk.services.s3.model.S3Exception;\n// Pre-flight: validate inputs that, if null, would yield NonTransient instead\nif (StringUtils.isBlank(bucketName) || StringUtils.isBlank(path)) {\n    throw new IllegalArgumentException(\"bucketName and path are required for getLocation\");\n}","typeGuard":null,"tryCatchPattern":"// TransientException => retriable; retry with exponential backoff\nint maxAttempts = 5;\nfor (int attempt = 1; attempt <= maxAttempts; attempt++) {\n    try {\n        return storage.getLocation(operation, payloadType, path);\n    } catch (TransientException e) {\n        if (attempt == maxAttempts) throw e;\n        Thread.sleep(Math.min(1000L * attempt, 5000L));\n    } catch (NonTransientException e) {\n        throw e; // config error — do not retry\n    }\n}","preventionTips":["Grant the IAM role s3:GetObject and s3:PutObject on the payload bucket.","Keep credentials/STS sessions valid long enough for presign + use windows.","Confirm region/endpoint match the bucket.","Wrap getLocation in a bounded retry for TransientException only."],"tags":["s3","aws","network","transient","storage","iam"],"backgroundTag":null,"analyzedSha":"cf7c3e4a8adfb158be778ab1ec525323c363cd3a","analyzedAt":"2026-08-14T03:33:19.897Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}