What causes "error decoding string field" in apache/beam?

Verify the input was produced by the matching Beam row encoder and is not truncated.

error decoding string field

Error message

error decoding string field

What it means

Wraps an error from DecodeStringUTF8 while decoding a string field: the length-prefixed UTF-8 string could not be read from the stream. Typically caused by EOF, a bad length prefix, or invalid alignment of the stream.

Solutions

Verify the input was produced by the matching Beam row encoder and is not truncated.
Re-encode the source data.
Check schema evolution compatibility (field order/count).
Inspect the bytes at the failure offset for corruption.

Defensive patterns

Strategy: try-catch

Try / catch

if err := decoderFn(rv, r); err != nil {
    if strings.Contains(err.Error(), "error decoding string field") {
        return fmt.Errorf("bad length-prefixed string in row: %w", err)
    }
    return err
}

Prevention

Ensure string fields are UTF-8 and written by the matching encoder.
Guard against reading partial files (check EOF before batch decode).
Keep schema versions aligned between producing and consuming jobs.
Test decoders against golden fixture blobs.

When it happens

Trigger: reflectDecodeString during row decoding when DecodeStringUTF8 fails — truncated data, corrupt length prefix, or decoding bytes not produced by the matching row encoder.

Common situations: Reading partial outputs from failed jobs, manually concatenated row blobs, schema drift between writer and reader.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/043143aa67294a15. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/graph/coder/row_decoder.go:220

		return errors.Wrap(err, "error decoding bool field")
	}
	rv.SetBool(v)
	return nil
}

func reflectDecodeByte(rv reflect.Value, r io.Reader) error {
	b, err := DecodeByte(r)
	if err != nil {
		return errors.Wrap(err, "error decoding single byte field")
	}
	rv.SetUint(uint64(b))
	return nil
}

func reflectDecodeString(rv reflect.Value, r io.Reader) error {
	v, err := DecodeStringUTF8(r)
	if err != nil {
		return errors.Wrap(err, "error decoding string field")
	}
	rv.SetString(v)
	return nil
}

func reflectDecodeInt(rv reflect.Value, r io.Reader) error {
	v, err := DecodeVarInt(r)
	if err != nil {
		return errors.Wrap(err, "error decoding varint field")
	}
	rv.SetInt(v)
	return nil
}

func reflectDecodeUint(rv reflect.Value, r io.Reader) error {
	v, err := DecodeVarUint64(r)
	if err != nil {
		return errors.Wrap(err, "error decoding varint field")

View on GitHub (pinned to 12126d8942)