{"record":{"id":"bcf15cfea324a175","repo":"apache/pulsar","slug":"offload-already-in-progress","errorCode":null,"errorMessage":"Offload already in progress","messagePattern":"Offload already in progress","errorType":"exception","errorClass":"org.apache.pulsar.broker.service.AlreadyRunningException","httpStatus":null,"severity":"warning","filePath":"pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java","lineNumber":4975,"sourceCode":"                        public void offloadComplete(Position pos, Object ctx) {\n                            Position impl = pos;\n                            log.info()\n                                    .attr(\"messageId\", messageId)\n                                    .log(\"Completed successfully offload operation at messageId\");\n                            promise.complete(new MessageIdImpl(impl.getLedgerId(), impl.getEntryId(), -1));\n                        }\n\n                        @Override\n                        public void offloadFailed(ManagedLedgerException exception, Object ctx) {\n                            log.warn()\n                                    .attr(\"messageId\", messageId)\n                                    .exception(exception)\n                                    .log(\"Failed offload operation at messageId\");\n                            promise.completeExceptionally(exception);\n                        }\n                    }, null);\n        } else {\n            throw new AlreadyRunningException(\"Offload already in progress\");\n        }\n    }\n\n    public synchronized OffloadProcessStatus offloadStatus() {\n        if (!currentOffload.isDone()) {\n            return OffloadProcessStatus.forStatus(LongRunningProcessStatus.Status.RUNNING);\n        } else {\n            try {\n                if (currentOffload.join() == MessageId.earliest) {\n                    return OffloadProcessStatus.forStatus(LongRunningProcessStatus.Status.NOT_RUN);\n                } else {\n                    return OffloadProcessStatus.forSuccess(currentOffload.join());\n                }\n            } catch (CancellationException | CompletionException e) {\n                log.warn(\"Failed to offload\");\n                return OffloadProcessStatus.forError(e.getMessage());\n            }\n        }","sourceCodeStart":4957,"sourceCodeEnd":4993,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L4957-L4993","documentation":"Like compaction, only one offload operation may run per topic; PersistentTopic tracks it with a currentOffload CompletableFuture. Calling triggerOffload while a previous offload is still in progress throws AlreadyRunningException('Offload already in progress'). The guard serializes storage offload work so two runs cannot offload the same ledgers concurrently.","triggerScenarios":"Calling admin topics().triggerOffload() (or internal offload APIs) while the topic's currentOffload future is not done — repeated manual triggers, overlapping scheduled offload jobs, or retry loops without checking offloadStatus().","commonSituations":"Automated offload scripts with intervals shorter than offload duration; operator re-clicking the offload trigger while a large-ledger offload runs; offloads slowed by slow object storage (S3/GCS) causing overlap of subsequent triggers.","solutions":["Check admin topics().offloadStatus() and trigger only when status is not RUNNING","Catch AlreadyRunningException and skip/back off, retrying after offloadStatus() reports completion","Throttle scheduled offload triggers based on measured offload duration; add per-topic locking","Speed up or investigate offload throughput (object storage latency, bandwidth limits) if it routinely overlaps"],"exampleFix":"// before\nadmin.topics().triggerOffload(topic, messageId); // throws if running\n// after\nif (!OffloadProcessStatus.Status.RUNNING.equals(admin.topics().offloadStatus(topic).status)) {\n    admin.topics().triggerOffload(topic, messageId);\n}","handlingStrategy":"try-catch","validationCode":"OffloadProcessStatus st = admin.topics().offloadStatus(topic);\nif (st.status == OffloadProcessStatus.Status.RUNNING) {\n    return; // skip trigger\n}","typeGuard":"boolean canTriggerOffload(OffloadProcessStatus s) {\n    return s != null && s.status != OffloadProcessStatus.Status.RUNNING;\n}","tryCatchPattern":"try {\n    admin.topics().triggerOffload(topic, messageId);\n} catch (PulsarAdminException e) {\n    if (e.getCause() instanceof AlreadyRunningException) {\n        log.info(\"Offload already running on {}\", topic);\n    } else throw e;\n}","preventionTips":["Check offloadStatus() before triggering","Serialize offload triggers per topic in automation","Account for object-storage latency when scheduling offloads"],"tags":["pulsar","broker","offload","storage","conflict"],"backgroundTag":"offload-already-running","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}