{"record":{"id":"1d29c7f46ced37dd","repo":"apache/druid","slug":"notenoughtemporarystoragefault","errorCode":null,"errorMessage":"NotEnoughTemporaryStorageFault","messagePattern":"NotEnoughTemporaryStorageFault","errorType":"error_code","errorClass":"MSQException","httpStatus":null,"severity":"critical","filePath":"multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerStorageParameters.java","lineNumber":108,"sourceCode":"   * @param tmpStorageBytesPerTask                  amount of disk space to be allocated per task for intermediate files.\n   * @param isIntermediateSuperSorterStorageEnabled whether intermediate super sorter storage is enabled\n   */\n  public static WorkerStorageParameters createInstance(\n      final long tmpStorageBytesPerTask,\n      final boolean isIntermediateSuperSorterStorageEnabled\n  )\n  {\n    if (!isIntermediateSuperSorterStorageEnabled || tmpStorageBytesPerTask == -1) {\n      return new WorkerStorageParameters(-1);\n    }\n\n    Preconditions.checkArgument(tmpStorageBytesPerTask > 0, \"Temporary storage bytes passed: [%s] should be > 0\", tmpStorageBytesPerTask);\n    long intermediateSuperSorterStorageMaxLocalBytes = computeUsableStorage(tmpStorageBytesPerTask);\n\n    log.info(\"Intermediate super sorter local storage size: %d bytes\", intermediateSuperSorterStorageMaxLocalBytes);\n\n    if (intermediateSuperSorterStorageMaxLocalBytes < MINIMUM_SUPER_SORTER_TMP_STORAGE_BYTES) {\n      throw new MSQException(\n          new NotEnoughTemporaryStorageFault(\n              calculateSuggestedMinTemporaryStorage(),\n              tmpStorageBytesPerTask\n          )\n      );\n    }\n\n    return new WorkerStorageParameters(intermediateSuperSorterStorageMaxLocalBytes);\n  }\n\n  private static long computeUsableStorage(long tmpStorageBytesPerTask)\n  {\n    return (long) (SUPER_SORTER_TMP_STORAGE_USABLE_FRACTION * (tmpStorageBytesPerTask - MINIMUM_BASIC_OPERATIONS_BYTES));\n  }\n\n  private static long calculateSuggestedMinTemporaryStorage()\n  {\n    return MINIMUM_BASIC_OPERATIONS_BYTES + (long) (MINIMUM_SUPER_SORTER_TMP_STORAGE_BYTES / SUPER_SORTER_TMP_STORAGE_USABLE_FRACTION);","sourceCodeStart":90,"sourceCodeEnd":126,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerStorageParameters.java#L90-L126","documentation":"WorkerStorageParameters.createInstance checks that the usable temporary storage derived from tmpStorageBytesPerTask meets MINIMUM_SUPER_SORTER_TMP_STORAGE_BYTES; if not, it throws MSQException(NotEnoughTemporaryStorageFault) with the suggested minimum and the configured value. This guards fault-tolerant MSQ queries that spill super-sorter data to local disk but have been allocated too little temporary space.","triggerScenarios":"Fault tolerance / durable shuffle enabled with tmpStorageBytesPerTask configured below the minimum super-sorter requirement; operator sets a small per-task temp storage quota in task context or worker config; druid.indexer task tmp dir volume is tiny.","commonSituations":"Enabling durable shuffle storage on workers with small local disks; setting per-task temp storage limits to protect shared disks but undersizing them; running on containers with small ephemeral storage mounts.","solutions":["Raise tmpStorageBytesPerTask to at least the value in NotEnoughTemporaryStorageFault.suggestedMinTemporaryStorage","Provision larger local scratch disk volumes for the peon/indexer tasks","Disable fault tolerance / durable shuffle if spilling is not needed for this query","Point the task temporary directory at a larger mount"],"exampleFix":"// before: task context\n// {\"druid.msq.indexer.tmp.storage.bytesPerTask\": 1000000}\n// after\n// {\"druid.msq.indexer.tmp.storage.bytesPerTask\": 10000000000} // >= suggested minimum","handlingStrategy":"validation","validationCode":"long usable = computeUsableStorage(tmpStorageBytesPerTask);\nif (usable < MINIMUM_SUPER_SORTER_TMP_STORAGE_BYTES) {\n  throw new IllegalArgumentException(\"Raise tmpStorageBytesPerTask to >= \" + calculateSuggestedMinTemporaryStorage());\n}","typeGuard":null,"tryCatchPattern":"try {\n  enableFaultTolerance(query);\n} catch (MSQException e) {\n  if (e.getFault() instanceof NotEnoughTemporaryStorageFault f) {\n    tmpStorageBytesPerTask = f.getSuggestedMinTemporaryStorage(); // retry with the suggestion\n  } else { throw e; }\n}","preventionTips":["Verify per-task free disk before enabling durable shuffle storage","Provision adequate scratch volumes on middle managers","Use the fault's suggestedMinTemporaryStorage to size configuration"],"tags":["msq","storage","temporary-disk"],"backgroundTag":"insufficient-disk-space","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}