thanos-io/thanos · error

read meta file

Error message

read meta file

What it means

Wraps a failure from ReadMetaFile(s.metadataFilePath) — reading the shipper's own thanos.json/meta.json tracking file that records which block ULIDs were already uploaded. If this file cannot be read (and is not an expected fresh-install missing file), AreAllBlocksUploaded cannot determine upload state and fails.

Solutions

1. Inspect the wrapped error; if the file is corrupt, delete it — the shipper will re-detect uploaded blocks via bucket Exists checks and rewrite it.
2. Fix file permissions/ownership so the sidecar process can read the data directory.
3. Check that the underlying filesystem is healthy and writable.
4. Restart the sidecar after repair so it rebuilds the upload manifest.

Example fix

// before
// read meta file: parse /data/thanos-meta.json: unexpected end of JSON input
// after
// $ rm /data/thanos-meta.json   # shipper will re-scan the bucket and recreate it
// $ systemctl restart thanos-sidecar

if f, err := os.Open(metadataFilePath); err != nil {
    if !os.IsNotExist(err) {
        log.Printf("shipper metadata unreadable, will need repair: %v", err)
    }
} else { f.Close() }

ok, err := shipper.AreAllBlocksUploaded()
if err != nil && strings.Contains(err.Error(), "read meta file") {
    // safe remediation: delete the corrupt metadata file; shipper rebuilds it from bucket Exists checks
    os.Remove(metadataFilePath)
    ok, err = shipper.AreAllBlocksUploaded()
}

When it happens

Trigger: Calling AreAllBlocksUploaded when the shipper's metadata file exists but is corrupt, unreadable (permissions), or the read fails with an I/O error other than the tolerated not-exist case.

Common situations: Truncated metadata file after power loss; wrong ownership/permissions on the data dir; read-only filesystem; metadata file deleted while blocks remain.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• write meta
• WAL dir is not accessible. Is this dir a TSDB directory? If…
• get all block meta.
• failed to read
• create meta fetcher

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/afd98053d0666049. Report an issue: GitHub.

Appendix: source

Thrown at pkg/shipper/shipper.go:321

	return nil
}

func (s *Shipper) AreAllBlocksUploaded() (bool, error) {
	s.mtx.RLock()
	defer s.mtx.RUnlock()

	metas, _, err := s.blockMetasFromOldest()
	if err != nil {
		return false, errors.Wrap(err, "get block metas from oldest")
	}

	if len(metas) == 0 {
		return true, nil
	}

	meta, err := ReadMetaFile(s.metadataFilePath)
	if err != nil {
		return false, errors.Wrap(err, "read meta file")
	}

	uploaded := make(map[ulid.ULID]struct{}, len(meta.Uploaded))
	for _, id := range meta.Uploaded {
		uploaded[id] = struct{}{}
	}

	for _, m := range metas {
		if _, ok := uploaded[m.ULID]; !ok {
			return false, nil
		}
	}

	return true, nil
}

// Sync performs a single synchronization, which ensures all non-compacted local blocks have been uploaded
// to the object bucket once.

View on GitHub (pinned to 35b8b99117)