{"record":{"id":"932acf63b32ae910","repo":"conductor-oss/conductor","slug":"unable-to-download-payload-from-external-storage-p","errorCode":null,"errorMessage":"Unable to download payload from external storage path: %s","messagePattern":"Unable to download payload from external storage path: (.+?)","errorType":"exception","errorClass":"NonTransientException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/com/netflix/conductor/core/utils/ExternalPayloadStorageUtils.java","lineNumber":76,"sourceCode":"    }\n\n    /**\n     * Download the payload from the given path.\n     *\n     * @param path the relative path of the payload in the {@link ExternalPayloadStorage}\n     * @return the payload object\n     * @throws NonTransientException in case of JSON parsing errors or download errors\n     */\n    @SuppressWarnings(\"unchecked\")\n    public Map<String, Object> downloadPayload(String path) {\n        try (InputStream inputStream = externalPayloadStorage.download(path)) {\n            return objectMapper.readValue(\n                    IOUtils.toString(inputStream, StandardCharsets.UTF_8), Map.class);\n        } catch (TransientException te) {\n            throw te;\n        } catch (Exception e) {\n            LOGGER.error(\"Unable to download payload from external storage path: {}\", path, e);\n            throw new NonTransientException(\n                    \"Unable to download payload from external storage path: \" + path, e);\n        }\n    }\n\n    /**\n     * Verify the payload size and upload to external storage if necessary.\n     *\n     * @param entity the task or workflow for which the payload is to be verified and uploaded\n     * @param payloadType the {@link PayloadType} of the payload\n     * @param <T> {@link TaskModel} or {@link WorkflowModel}\n     * @throws NonTransientException in case of JSON parsing errors or upload errors\n     * @throws TerminateWorkflowException if the payload size is bigger than permissible limit as\n     *     per {@link ConductorProperties}\n     */\n    public <T> void verifyAndUpload(T entity, PayloadType payloadType) {\n        if (!shouldUpload(entity, payloadType)) return;\n\n        long threshold = 0L;","sourceCodeStart":58,"sourceCodeEnd":94,"githubUrl":"https://github.com/conductor-oss/conductor/blob/cf7c3e4a8adfb158be778ab1ec525323c363cd3a/core/src/main/java/com/netflix/conductor/core/utils/ExternalPayloadStorageUtils.java#L58-L94","documentation":"Thrown by ExternalPayloadStorageUtils.downloadPayload when downloading and deserializing a payload from external storage fails for any non-transient reason — e.g. the object doesn't exist at the path, the downloaded content is not valid JSON, or the storage backend returns an error. TransientException is re-thrown as-is; everything else is wrapped in NonTransientException, meaning retries will not help.","triggerScenarios":"Calling downloadPayload with a path that does not exist in external storage, points to corrupted/truncated JSON, or triggers a storage backend error (e.g. S3 403 permission denied). Occurs when the decider loads a task or workflow whose input/output payload was externalized to external storage and the external object is missing or corrupt.","commonSituations":"External storage bucket/object was deleted or expired (TTL). Permissions changed on the storage backend (S3 IAM policy revoked). The payload path is stale after a storage migration. JSON corruption from a partial write or encoding mismatch. External storage misconfigured (wrong endpoint, credentials expired).","solutions":["Verify the external storage path exists and is accessible — check the S3/GCP/Azure bucket for the object at the given path.","Check storage backend credentials and permissions in the Conductor configuration (access key, secret, IAM role).","If the object was deleted by TTL or lifecycle policy, increase the retention period or re-run the workflow to regenerate the payload.","Inspect the server log for the underlying cause exception (logged at LOGGER.error with the path) to determine if it's a permissions issue, missing object, or JSON parse error.","If the payload is permanently lost, terminate and re-run the affected workflow instance."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Verify external storage path exists before downloading (if backend supports checks)\n// This is backend-specific; for S3 you would use headObject\n// Generic pre-check:\nif (path == null || path.isEmpty()) {\n    throw new IllegalArgumentException(\"External payload path is null or empty\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    Map<String, Object> payload = externalPayloadStorageUtils.downloadPayload(path);\n} catch (NonTransientException e) {\n    LOGGER.error(\"Payload at {} is missing or corrupt — workflow may need to be re-run\", path, e);\n    // Mark the task/workflow as failed or trigger re-execution\n    throw e;\n}","preventionTips":["Configure adequate TTL/retention on external storage objects to prevent premature deletion.","Verify storage credentials and permissions before deploying workflows that use external payload storage.","Monitor external storage health and alert on missing objects.","Keep external storage bucket lifecycle policies from deleting active workflow payloads."],"tags":["external-storage","payload","non-transient","network","serialization"],"backgroundTag":null,"analyzedSha":"cf7c3e4a8adfb158be778ab1ec525323c363cd3a","analyzedAt":"2026-08-14T03:33:19.897Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}