apache/druid · error · ISE

Unable to serialize [ ], max size bytes is [ ], but need at…

Error message

Unable to serialize [%s], max size bytes is [%s], but need at least [%s] bytes to write entire value

What it means

ExprEval.serialize writes the value into a byte buffer using the type's serialization strategy and a maxSizeBytes limit. When the strategy's write returns a negative value, the value did not fit within the allotted bytes, and an ISE is thrown reporting the type, the byte budget, and the additional bytes needed.

Solutions

1. Increase the maxSizeBytes parameter for the serialization context
2. Truncate or reduce the value's size before serialization (e.g. substring long strings, limit array length)
3. Adjust the column type (e.g. smaller representation) or the ingestion spec so values fit

Example fix

// before
exprEval.serialize(buffer, offset, type, 128);
// after
exprEval.serialize(buffer, offset, type, 1024); // size budget raised to fit value

// ensure the value fits before serializing
if (strategy.calcSizeBytes(value) > maxSizeBytes) {
  throw new IllegalArgumentException("value exceeds maxSizeBytes=" + maxSizeBytes);
}

try {
  eval.serialize(buffer, offset, type, maxSizeBytes);
} catch (IllegalStateException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Unable to serialize")) {
    maxSizeBytes = computeRequiredBytes(value); // retry with a bigger budget
  } else { throw e; }
}

When it happens

Trigger: Serializing an expression value whose byte representation exceeds maxSizeBytes, e.g. very long strings, huge nested arrays, or oversized numbers in columnar/agg serialization paths.

Common situations: A dimension or output column's real-world size exceeding the configured max byte size; migrating data with larger values into a setup tuned for smaller strings; misconfigured maxSizeBytes in aggregators/serializers.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• cannot stringify array type
• Not supported
• argument should be an identifier expression. Use array()…
• bucketSize must be a power of two but was[%,d]
• bucketSize must be a power of two but was[%,d]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/c7f1fee8f1a8d691. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/math/expr/ExprEval.java:136

          TypeStrategies.writeNotNullNullableLong(buffer, offset, eval.asLong());
        }
        break;
      case DOUBLE:
        if (eval.value() == null) {
          TypeStrategies.writeNull(buffer, offset);
        } else {
          TypeStrategies.writeNotNullNullableDouble(buffer, offset, eval.asDouble());
        }
        break;
      default:
        final NullableTypeStrategy strategy = type.getNullableStrategy();
        // if the types don't match, cast it so things don't get weird
        if (type.equals(eval.type())) {
          eval = eval.castTo(type);
        }
        int written = strategy.write(buffer, offset, eval.value(), maxSizeBytes);
        if (written < 0) {
          throw new ISE(
              "Unable to serialize [%s], max size bytes is [%s], but need at least [%s] bytes to write entire value",
              type.asTypeString(),
              maxSizeBytes,
              maxSizeBytes - written
          );
        }
    }
  }

  /**
   * Converts a List to an appropriate array type, optionally doing some conversion to make multi-valued strings
   * consistent across selector types, which are not consistent in treatment of null, [], and [null].
   *
   * If homogenizeMultiValueStrings is true, null and [] will be converted to [null], otherwise they will retain
   */
  @Nullable
  public static NonnullPair<ExpressionType, Object[]> coerceListToArray(@Nullable List<?> val, boolean homogenizeMultiValueStrings)
  {

View on GitHub (pinned to 9b90983fd2)