What causes "data too short for header: %d bytes" in microsoft/typescript-go?

If the data arrived as base64 (as in API node handles), decode it with base64.StdEncoding first

data too short for header: %d bytes

Error message

data too short for header: %d bytes

What it means

The binary AST decoder requires at least HeaderSize bytes of fixed header before anything can be read; the supplied buffer is shorter. This is the first sanity gate in newASTDecoder and fires before version or offset checks.

Source

Thrown at internal/api/encoder/decoder.go:55

	}
	if node.Kind != ast.KindSourceFile {
		return nil, fmt.Errorf("expected SourceFile root, got %v", node.Kind)
	}
	return node.AsSourceFile(), nil
}

// DecodeNodes decodes binary-encoded AST data into a tree of *ast.Node objects.
func DecodeNodes(data []byte) (*ast.Node, error) {
	d, err := newASTDecoder(data)
	if err != nil {
		return nil, err
	}
	return d.decode()
}

func newASTDecoder(data []byte) (*astDecoder, error) {
	if len(data) < HeaderSize {
		return nil, fmt.Errorf("data too short for header: %d bytes", len(data))
	}
	version := data[HeaderOffsetMetadata+3]
	if version != ProtocolVersion {
		return nil, fmt.Errorf("unsupported protocol version %d (expected %d)", version, ProtocolVersion)
	}

	strTable := readLE32(data, HeaderOffsetStringOffsets)
	strData := readLE32(data, HeaderOffsetStringData)
	extData := readLE32(data, HeaderOffsetExtendedData)
	nodeOff := readLE32(data, HeaderOffsetNodes)

	dataLen := uint32(len(data))

	// Validate that all offsets are within the buffer.
	if strTable > dataLen || strData > dataLen || extData > dataLen || nodeOff > dataLen {
		return nil, fmt.Errorf("invalid AST header offsets: offsets exceed data length (%d)", dataLen)
	}

View on GitHub (pinned to 1bcfa18d79)

Solutions

If the data arrived as base64 (as in API node handles), decode it with base64.StdEncoding first
Guard with len(data) >= encoder.HeaderSize before calling and reject upstream
Re-encode the source file to regenerate a valid blob
Verify the transport preserved payload length (content-length or framing mismatch)

Example fix

// before
node, err := encoder.DecodeSourceFile([]byte(b64String))

// after
raw, err := base64.StdEncoding.DecodeString(b64String)
if err != nil { return err }
node, err := encoder.DecodeSourceFile(raw)

Defensive patterns

Strategy: validation

Validate before calling

if len(data) < encoder.HeaderSize {
	return fmt.Errorf("encoded AST too short: %d bytes (need %d)", len(data), encoder.HeaderSize)
}
sf, err := encoder.DecodeSourceFile(data)

Prevention

Base64-decode API payloads before DecodeSourceFile
Check lengths on cache reads and reject short blobs upstream
Verify transport framing preserves payload length

When it happens

Trigger: Passing an empty or near-empty slice; passing base64 text instead of decoded bytes; passing raw TypeScript source; a truncated cache file or a zero-length read from transport.

Common situations: Forgetting base64 decode before DecodeSourceFile (the API layer decodes base64 handles itself); cache files truncated by disk-full or interrupted writes; slicing a larger buffer with wrong indices.

Related errors

AI-assisted analysis of microsoft/typescript-go@1bcfa18d79 (2026-08-16). Data as JSON: /api/errors/4b2b5b44a9a53db4. Report an issue: GitHub.