openzipkin/zipkin · error · IllegalArgumentException

Cannot gunzip spans

Error message

Cannot gunzip spans

What it means

UnzippingBytesRequestConverter.convertRequest throws IllegalArgumentException ('Cannot gunzip spans') when a POST to the Zipkin HTTP collector declares a gzip Content-Encoding but Armeria's gzip decoder produced an empty body, which the decoder does on failure. This indicates the client sent malformed or truncated gzip data; the request is rejected instead of being treated as an empty span list.

Source

Thrown at zipkin-server/src/main/java/zipkin2/server/internal/ZipkinHttpCollector.java:238

  @Override public void onError(Throwable t) {
    completeExceptionally(t);
  }
}

final class UnzippingBytesRequestConverter {

  static HttpData convertRequest(ServiceRequestContext ctx, AggregatedHttpRequest request) {
    ZipkinHttpCollector.metrics.incrementMessages();
    String encoding = request.headers().get(HttpHeaderNames.CONTENT_ENCODING);
    HttpData content = request.content();
    if (!content.isEmpty() && encoding != null && encoding.contains("gzip")) {
      content = StreamDecoderFactory.gzip().newDecoder(ctx.alloc()).decode(content);
      // The implementation of the armeria decoder is to return an empty body on failure
      if (content.isEmpty()) {
        ZipkinHttpCollector.maybeLog("Malformed gzip body", ctx, request);
        content.close();
        throw new IllegalArgumentException("Cannot gunzip spans");
      }
    }

    if (content.isEmpty()) ZipkinHttpCollector.maybeLog("Empty POST body", ctx, request);
    if (content.length() == 2 && "[]".equals(content.toStringAscii())) {
      ZipkinHttpCollector.maybeLog("Empty JSON list POST body", ctx, request);
      content.close();
      content = HttpData.empty();
    }

    ZipkinHttpCollector.metrics.incrementBytes(content.length());
    return content;
  }
}

View on GitHub (pinned to 878ce2a1fa)

Solutions

1. Fix the sender to send actual gzip-compressed bytes when the header is present
2. Remove proxies/interceptors that add or strip Content-Encoding incorrectly
3. Test with curl --data-binary @spans.gz --header 'Content-Encoding: gzip' --header 'Content-Type: application/json' to confirm valid gzip

Example fix

# before
curl -X POST http://zipkin:9411/api/v2/spans \
  -H 'Content-Encoding: gzip' \
  --data-binary @spans.json   # plain JSON with gzip header -> error

# after
gzip -c spans.json > spans.gz
curl -X POST http://zipkin:9411/api/v2/spans \
  -H 'Content-Encoding: gzip' -H 'Content-Type: application/json' \
  --data-binary @spans.gz

// sender side: only set the header when the body is actually gzipped
byte[] body = json.getBytes(UTF_8);
boolean gzipped = false; // set true only when you gzip yourself
request.header("Content-Encoding", gzipped ? "gzip" : "identity").body(body);

catch (IllegalArgumentException e) if 'Cannot gunzip spans' -> log the offending payload size/encoding headers, fix the sender's compression logic, and drop the batch (do not retry the same bytes)

When it happens

Trigger: POST /api/v2/spans with Content-Encoding: gzip where the body is not valid gzip (e.g. already-decoded JSON double-gzipped pipeline, cut-off payload from a proxy, or raw bytes with the header added manually).

Common situations: A proxy or SDK applies the gzip header without actually compressing; senders re-compress already-compressed bodies; load balancers truncate large compressed batches; curl with --header 'Content-Encoding: gzip' but uncompressed data.

Related errors

• metrics == null
• loggingClass == null
• storage == null
• metrics == null
• sampler == null

AI-assisted analysis of openzipkin/zipkin@878ce2a1fa (2026-08-14). Data as JSON: /api/errors/448d0e63fc656c3f. Report an issue: GitHub.