apache/beam · error

error encoding byte: %v

Error message

error encoding byte: %v

What it means

EncodeByte writes a single raw byte to an io.Writer using ioutilx.WriteUnsafe, and wraps any write failure as 'error encoding byte: %v'. It is part of the Beam Go SDK core coder package used when serializing byte-typed elements in pipelines.

Source

Thrown at sdks/go/pkg/beam/core/graph/coder/bytes.go:30

// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package coder

import (
	"fmt"
	"io"

	"github.com/apache/beam/sdks/v2/go/pkg/beam/core/util/ioutilx"
	"github.com/apache/beam/sdks/v2/go/pkg/beam/internal/errors"
)

// EncodeByte encodes a single byte.
func EncodeByte(v byte, w io.Writer) error {
	// Encoding: raw byte.
	if _, err := ioutilx.WriteUnsafe(w, []byte{v}); err != nil {
		return fmt.Errorf("error encoding byte: %v", err)
	}
	return nil
}

// DecodeByte decodes a single byte.
func DecodeByte(r io.Reader) (byte, error) {
	// Encoding: raw byte
	var b [1]byte
	if err := ioutilx.ReadNBufUnsafe(r, b[:]); err != nil {
		if err == io.EOF {
			return 0, err
		}
		return 0, errors.Wrap(err, "error decoding byte")
	}
	return b[0], nil
}

// EncodeBytes encodes a []byte with a length prefix per the beam protocol.

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the wrapped cause; fix the underlying writer/stream failure (re-establish the data channel, reopen the file).
2. Ensure the io.Writer passed to the coder is valid and open for the lifetime of encoding.
3. Retry the failing bundle if this occurred during distributed execution (runners usually retry automatically).
4. Check that the writer is not nil; WriteUnsafe on a nil writer will fail.

Example fix

// before
var w io.Writer // nil
 coder.EncodeByte(0x1, w)
// after
var w io.Writer = &bytes.Buffer{}
if err := coder.EncodeByte(0x1, w); err != nil { log.Fatal(err) }

if w == nil { return errors.New("writer must be non-nil before encoding bytes") }

if err := coder.EncodeByte(v, w); err != nil {
	if errors.Is(err, io.ErrClosedPipe) || errors.Is(err, io.ErrUnexpectedEOF) {
		// re-open stream and retry the record
	}
	return err
}

When it happens

Trigger: Any pipeline element encoding path that calls bytes.EncodeByte when the underlying io.Writer fails: closed stream, broken connection to a runner/harness, or a full/closed buffer.

Common situations: A runner data channel (gRPC stream) breaks mid-bundle so the sink writer errors; tests passing a nil or failing writer; writing to a closed pipe during local pipeline execution.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• WriteSimpleRowHeader a 0 length nils bit field: %v
• varint too long
• error encoding pane %v: non-speculative index value must be
• number of fields is less than byte array %v < %v
• ReadSimpleRowHeader field count: %v, %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6cb0a74be4568862. Report an issue: GitHub.