{"record":{"id":"1e7bb26b986fc4ed","repo":"VictoriaMetrics/VictoriaMetrics","slug":"too-big-data-size-d-it-mustn-t-exceed-d-bytes","errorCode":null,"errorMessage":"too big data size: %d; it mustn't exceed %d bytes","messagePattern":"too big data size: (.+?); it mustn't exceed (.+?) bytes","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"lib/vmselectapi/server.go","lineNumber":397,"sourceCode":"\t\treturn fmt.Errorf(\"cannot unmarshal SearchQuery: %w\", err)\n\t}\n\tif len(tail) > 0 {\n\t\treturn fmt.Errorf(\"unexpected non-zero tail left after unmarshaling SearchQuery: (len=%d) %q\", len(tail), tail)\n\t}\n\treturn nil\n}\n\nfunc (ctx *vmselectRequestCtx) readDataBufBytes(maxDataSize int) error {\n\tctx.sizeBuf = bytesutil.ResizeNoCopyMayOverallocate(ctx.sizeBuf, 8)\n\tif _, err := io.ReadFull(ctx.bc, ctx.sizeBuf); err != nil {\n\t\tif err == io.EOF {\n\t\t\treturn err\n\t\t}\n\t\treturn fmt.Errorf(\"cannot read data size: %w\", err)\n\t}\n\tdataSize := encoding.UnmarshalUint64(ctx.sizeBuf)\n\tif dataSize > uint64(maxDataSize) {\n\t\treturn fmt.Errorf(\"too big data size: %d; it mustn't exceed %d bytes\", dataSize, maxDataSize)\n\t}\n\tctx.dataBuf = bytesutil.ResizeNoCopyMayOverallocate(ctx.dataBuf, int(dataSize))\n\tif dataSize == 0 {\n\t\treturn nil\n\t}\n\tif n, err := io.ReadFull(ctx.bc, ctx.dataBuf); err != nil {\n\t\treturn fmt.Errorf(\"cannot read data with size %d: %w; read only %d bytes\", dataSize, err, n)\n\t}\n\treturn nil\n}\n\nfunc (ctx *vmselectRequestCtx) readBool() (bool, error) {\n\tctx.dataBuf = bytesutil.ResizeNoCopyMayOverallocate(ctx.dataBuf, 1)\n\tif _, err := io.ReadFull(ctx.bc, ctx.dataBuf); err != nil {\n\t\tif err == io.EOF {\n\t\t\treturn false, err\n\t\t}\n\t\treturn false, fmt.Errorf(\"cannot read bool: %w\", err)","sourceCodeStart":379,"sourceCodeEnd":415,"githubUrl":"https://github.com/VictoriaMetrics/VictoriaMetrics/blob/5079fb58f1e8e62113f90c945ad71586c797d770/lib/vmselectapi/server.go#L379-L415","documentation":"Returned by vmselectRequestCtx.readDataBufBytes when the 8-byte size header was read successfully but declares a data size larger than the caller's maxDataSize limit (e.g. maxSearchQuerySize = 5MiB). This is a deliberate guard against oversized/allocation-bombing packets; the server refuses to allocate the buffer and aborts the request.","triggerScenarios":"A client sends a length-prefixed packet whose declared dataSize exceeds the endpoint's limit: a SearchQuery packet >5MiB, or oversized packets for processRequest, processRegisterMetricNames, processLabelValues, processTagValueSuffixes, processTSDBStatus.","commonSituations":"Corrupted/garbage size header (e.g. desynced stream reading payload bytes as a size); runaway client buffers serialized whole; adversarial clients sending huge declared sizes; version mismatches with larger packet formats.","solutions":["Fix client-side stream desync: a bogus huge size usually means a previous packet was written with the wrong length; re-align the protocol framing.","Reduce request size (split large label-value or metadata requests into batches).","Check for byte-order bugs in the client's size encoding (big-endian vs little-endian produces huge values).","If legitimately larger packets are needed, raise the max size (maxSearchQuerySize) and rebuild, understanding the memory trade-off.","Investigate the sending client for corruption or malicious traffic; consider network ACLs."],"exampleFix":"// before: big-endian size confuses server\nbinary.BigEndian.PutUint64(hdr, uint64(len(payload)))\n// after\nvar hdr [8]byte\nbinary.LittleEndian.PutUint64(hdr[:], uint64(len(payload)))\nconn.Write(hdr[:])\nconn.Write(payload)","handlingStrategy":"validation","validationCode":"// client side: enforce the same limits as the server\nconst maxSearchQuerySize = 5 * 1024 * 1024\nif uint64(len(payload)) > maxSearchQuerySize {\n    return fmt.Errorf(\"payload %d exceeds server limit %d\", len(payload), maxSearchQuerySize)\n}","typeGuard":null,"tryCatchPattern":"if err := ctx.readDataBufBytes(max); err != nil {\n    if strings.Contains(err.Error(), \"too big data size\") {\n        return ErrPacketTooLarge // permanent; do not retry identical request\n    }\n    return err\n}","preventionTips":["Enforce packet-size limits client-side mirroring server constants","Check byte order of the size encoding; endian bugs yield huge sizes","Split large requests (label values, metadata) into batches","Treat this as a permanent error: fix framing or size, never blind-retry","Watch for stream desync: a garbage size usually means a previous wrong-length write"],"tags":["protocol","framing","validation","dos-protection"],"backgroundTag":"packet-size-limit-exceeded","analyzedSha":"5079fb58f1e8e62113f90c945ad71586c797d770","analyzedAt":"2026-09-03T18:10:26.153Z","contentChangedAt":"2026-09-03T18:10:26.153Z","schemaVersion":2},"datasetVersion":"2026-09-08T15:18:49.778Z"}