{"record":{"id":"b3d1649b7fd535a1","repo":"ruvnet/RuView","slug":"frame-too-short-need-self-header-size-bytes-go","errorCode":null,"errorMessage":"Frame too short: need {self.HEADER_SIZE} bytes, got {len(raw_data)}","messagePattern":"Frame too short: need (.+?) bytes, got (.+?)","errorType":"exception","errorClass":"CSIParseError","httpStatus":null,"severity":"error","filePath":"archive/v1/src/hardware/csi_extractor.py","lineNumber":189,"sourceCode":"    PPDU_HE_MU = 2\n    PPDU_HE_TB = 3\n    PPDU_UNKNOWN = 0xFF\n    _PPDU_NAMES = {0: 'ht_legacy', 1: 'he_su', 2: 'he_mu', 3: 'he_tb', 0xFF: 'unknown'}\n\n    def parse(self, raw_data: bytes) -> CSIData:\n        \"\"\"Parse an ADR-018 binary frame into CSIData.\n\n        Args:\n            raw_data: Raw binary frame bytes.\n\n        Returns:\n            Parsed CSI data with amplitude/phase arrays shaped (n_antennas, n_subcarriers).\n\n        Raises:\n            CSIParseError: If frame is too short, has invalid magic, or malformed I/Q data.\n        \"\"\"\n        if len(raw_data) < self.HEADER_SIZE:\n            raise CSIParseError(\n                f\"Frame too short: need {self.HEADER_SIZE} bytes, got {len(raw_data)}\"\n            )\n\n        magic, node_id, n_antennas, n_subcarriers, freq_mhz, sequence, rssi_u8, noise_u8, \\\n            ppdu_byte, flags_byte = struct.unpack_from(self.HEADER_FMT, raw_data, 0)\n\n        if magic != self.MAGIC:\n            raise CSIParseError(\n                f\"Invalid magic: expected 0x{self.MAGIC:08X}, got 0x{magic:08X}\"\n            )\n\n        # Convert unsigned bytes to signed i8\n        rssi = rssi_u8 if rssi_u8 < 128 else rssi_u8 - 256\n        noise_floor = noise_u8 if noise_u8 < 128 else noise_u8 - 256\n\n        iq_count = n_antennas * n_subcarriers\n        iq_bytes = iq_count * 2\n        expected_len = self.HEADER_SIZE + iq_bytes","sourceCodeStart":171,"sourceCodeEnd":207,"githubUrl":"https://github.com/ruvnet/RuView/blob/4685618388a5e49fad5b3005806f3bdd6a7c25c3/archive/v1/src/hardware/csi_extractor.py#L171-L207","documentation":"ESP32BinaryParser (ADR-018 frames, selected via config parser_format='binary') rejects any buffer shorter than the 20-byte header (HEADER_SIZE=20, format '<IBBHIIBBBB': magic, node_id, antennas, subcarriers, freq, seq, rssi, noise, ppdu, flags) before attempting struct.unpack. This is a framing problem: not enough bytes exist to even read the header.","triggerScenarios":"parse() on a fragment (e.g. first 8 bytes of a frame from a chunked TCP read); an empty buffer after stream EOF; handing ASCII 'CSI_DATA' text to the binary parser; a UDP datagram split by an intermediary.","commonSituations":"Starting to read mid-frame so the first buffer is the tail of a previous frame; assuming TCP byte-stream reads align with frame boundaries; collector slicing the stream at a fixed size unrelated to frame length.","solutions":["Buffer incoming bytes and call parse only when at least HEADER_SIZE (20) bytes are available; for UDP, parse each datagram as one frame","Resync the stream: scan for the 4-byte little-endian magic 0xC5110001 before treating bytes as a frame start","Confirm parser_format matches what the firmware sends (text CSV vs binary ADR-018)"],"exampleFix":"# before\nframe = await reader.read(64)  # may return a partial frame\ndata = parser.parse(frame)  # CSIParseError: Frame too short\n\n# after\nbuf = bytearray()\nwhile len(buf) < parser.HEADER_SIZE:\n    buf += await reader.read(parser.HEADER_SIZE - len(buf))\n# then read the I/Q payload declared by the header before parsing","handlingStrategy":"validation","validationCode":"HEADER_SIZE = ESP32BinaryParser.HEADER_SIZE  # 20\n\nif len(buf) < HEADER_SIZE:\n    # not a complete frame yet: keep buffering / skip fragment\n    continue\ndata = parser.parse(buf)","typeGuard":"def is_frame_sized(buf: bytes) -> bool:\n    return len(buf) >= ESP32BinaryParser.HEADER_SIZE","tryCatchPattern":"try:\n    data = parser.parse(frame_bytes)\nexcept CSIParseError as e:\n    logger.debug('dropping short binary frame: %s', e)\n    resync_buffer()","preventionTips":["For UDP, parse each datagram as one frame; for byte streams, buffer until at least 20 bytes (then the full declared length)","Never assume TCP read sizes align with frame boundaries — accumulate and frame on magic","Log dropped fragments with their lengths: a run of small fragments means the framing loop, not the transport, is wrong"],"tags":["esp32","binary","parsing","framing","adr-018"],"backgroundTag":null,"analyzedSha":"4685618388a5e49fad5b3005806f3bdd6a7c25c3","analyzedAt":"2026-08-16T06:09:40.886Z","schemaVersion":2},"datasetVersion":"2026-08-16T08:17:34.114Z"}