{"record":{"id":"c7f1fee8f1a8d691","repo":"apache/druid","slug":"unable-to-serialize-s-max-size-bytes-is-s","errorCode":null,"errorMessage":"Unable to serialize [%s], max size bytes is [%s], but need at least [%s] bytes to write entire value","messagePattern":"Unable to serialize \\[(.+?)\\], max size bytes is \\[(.+?)\\], but need at least \\[(.+?)\\] bytes to write entire value","errorType":"exception","errorClass":"ISE","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/math/expr/ExprEval.java","lineNumber":136,"sourceCode":"          TypeStrategies.writeNotNullNullableLong(buffer, offset, eval.asLong());\n        }\n        break;\n      case DOUBLE:\n        if (eval.value() == null) {\n          TypeStrategies.writeNull(buffer, offset);\n        } else {\n          TypeStrategies.writeNotNullNullableDouble(buffer, offset, eval.asDouble());\n        }\n        break;\n      default:\n        final NullableTypeStrategy strategy = type.getNullableStrategy();\n        // if the types don't match, cast it so things don't get weird\n        if (type.equals(eval.type())) {\n          eval = eval.castTo(type);\n        }\n        int written = strategy.write(buffer, offset, eval.value(), maxSizeBytes);\n        if (written < 0) {\n          throw new ISE(\n              \"Unable to serialize [%s], max size bytes is [%s], but need at least [%s] bytes to write entire value\",\n              type.asTypeString(),\n              maxSizeBytes,\n              maxSizeBytes - written\n          );\n        }\n    }\n  }\n\n  /**\n   * Converts a List to an appropriate array type, optionally doing some conversion to make multi-valued strings\n   * consistent across selector types, which are not consistent in treatment of null, [], and [null].\n   *\n   * If homogenizeMultiValueStrings is true, null and [] will be converted to [null], otherwise they will retain\n   */\n  @Nullable\n  public static NonnullPair<ExpressionType, Object[]> coerceListToArray(@Nullable List<?> val, boolean homogenizeMultiValueStrings)\n  {","sourceCodeStart":118,"sourceCodeEnd":154,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/math/expr/ExprEval.java#L118-L154","documentation":"ExprEval.serialize writes the value into a byte buffer using the type's serialization strategy and a maxSizeBytes limit. When the strategy's write returns a negative value, the value did not fit within the allotted bytes, and an ISE is thrown reporting the type, the byte budget, and the additional bytes needed.","triggerScenarios":"Serializing an expression value whose byte representation exceeds maxSizeBytes, e.g. very long strings, huge nested arrays, or oversized numbers in columnar/agg serialization paths.","commonSituations":"A dimension or output column's real-world size exceeding the configured max byte size; migrating data with larger values into a setup tuned for smaller strings; misconfigured maxSizeBytes in aggregators/serializers.","solutions":["Increase the maxSizeBytes parameter for the serialization context","Truncate or reduce the value's size before serialization (e.g. substring long strings, limit array length)","Adjust the column type (e.g. smaller representation) or the ingestion spec so values fit"],"exampleFix":"// before\nexprEval.serialize(buffer, offset, type, 128);\n// after\nexprEval.serialize(buffer, offset, type, 1024); // size budget raised to fit value","handlingStrategy":"validation","validationCode":"// ensure the value fits before serializing\nif (strategy.calcSizeBytes(value) > maxSizeBytes) {\n  throw new IllegalArgumentException(\"value exceeds maxSizeBytes=\" + maxSizeBytes);\n}","typeGuard":null,"tryCatchPattern":"try {\n  eval.serialize(buffer, offset, type, maxSizeBytes);\n} catch (IllegalStateException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Unable to serialize\")) {\n    maxSizeBytes = computeRequiredBytes(value); // retry with a bigger budget\n  } else { throw e; }\n}","preventionTips":["Size maxSizeBytes to the largest realistic value","Bound string lengths and array sizes at ingestion","Load-test serialization with production-sized data"],"tags":["serialization","expressions","size-limit"],"backgroundTag":"payload-too-large","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}