thanos-io/thanos · error

read file

Error message

read file: %s

What it means

Inside metadata.ReadMarker, when io.ReadAll fails while draining the marker file's reader, the error is wrapped with "read file: <path>". The Get succeeded but streaming the marker bytes from the object storage failed mid-read.

Solutions

1. Retry the read — this is usually a transient network failure; consider bucket client retry settings.
2. Check network stability/proxy timeouts between the client and object storage.
3. Inspect the wrapped cause for the provider-specific error and tune the bucket client (timeouts, max idle conns).

if err := metadata.ReadMarker(ctx, logger, bkt, id, mark); err != nil {
	if strings.Contains(err.Error(), "read file:") {
		// transient stream failure: retry with backoff
		return retry(ctx, func() error { return metadata.ReadMarker(ctx, logger, bkt, id, mark) })
	}
	return err
}

When it happens

Trigger: ReadMarker obtains a bucket reader successfully, then io.ReadAll(r) errors due to connection reset/timeout while streaming, TLS termination, or an underlying reader that fails partway through the download.

Common situations: Flaky network between the client and S3/GCS/Azure; long-running fetcher operations hitting idle connection timeouts; proxies or load balancers cutting connections mid-body.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• read meta file
• get header from object storage of
• get file
• read range
• sync before first pass of downsampling

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/3d4a2c50b3418d6f. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/metadata/markers.go:132

}

func (n *NoDownsampleMark) markerFilename() string { return NoDownsampleMarkFilename }

// ReadMarker reads the given mark file from <dir>/<marker filename>.json in bucket.
func ReadMarker(ctx context.Context, logger log.Logger, bkt objstore.InstrumentedBucketReader, dir string, marker Marker) error {
	markerFile := path.Join(dir, marker.markerFilename())
	r, err := bkt.ReaderWithExpectedErrs(bkt.IsObjNotFoundErr).Get(ctx, markerFile)
	if err != nil {
		if bkt.IsObjNotFoundErr(err) {
			return ErrorMarkerNotFound
		}
		return errors.Wrapf(err, "get file: %s", markerFile)
	}
	defer runutil.CloseWithLogOnErr(logger, r, "close bkt marker reader")

	metaContent, err := io.ReadAll(r)
	if err != nil {
		return errors.Wrapf(err, "read file: %s", markerFile)
	}

	if err := json.Unmarshal(metaContent, marker); err != nil {
		return errors.Wrapf(ErrorUnmarshalMarker, "file: %s; err: %v", markerFile, err.Error())
	}
	switch marker.markerFilename() {
	case NoCompactMarkFilename:
		if version := marker.(*NoCompactMark).Version; version != NoCompactMarkVersion1 {
			return errors.Errorf("unexpected no-compact-mark file version %d, expected %d", version, NoCompactMarkVersion1)
		}
	case NoDownsampleMarkFilename:
		if version := marker.(*NoDownsampleMark).Version; version != NoDownsampleMarkVersion1 {
			return errors.Errorf("unexpected no-downsample-mark file version %d, expected %d", version, NoDownsampleMarkVersion1)
		}
	case DeletionMarkFilename:
		if version := marker.(*DeletionMark).Version; version != DeletionMarkVersion1 {
			return errors.Errorf("unexpected deletion-mark file version %d, expected %d", version, DeletionMarkVersion1)
		}

View on GitHub (pinned to 35b8b99117)