thanos-io/thanos · error

read next field from parquet metadata file

Error message

read next field from parquet metadata file: %v

What it means

Wraps easyproto FieldContext.NextField while sequentially decoding the parquet conversion metadata protobuf: the byte stream is not a well-formed protobuf (truncated mid-field or corrupted framing), so block IDs of parquet-converted blocks cannot be enumerated.

Solutions

1. Verify the object is a valid parquet metadata protobuf file
2. Re-generate or re-upload the metadata object
3. Skip/ignore corrupt metadata paths

meta, err := bkt.Get(ctx, path)
if err == nil {
    if n, _ := io.ReadAll(meta); len(n) == 0 {
        return fmt.Errorf("parquet metadata %s is empty/corrupt", path)
    }
}

if err := decodeMeta(content); err != nil {
    log.Printf("skipping corrupt parquet metadata %s: %v", path, err)
    // skip file instead of failing the whole scan
}

When it happens

Trigger: fc.NextField(content) returns an error in the decoding loop — the file content is truncated, corrupt, or not the expected protobuf format (e.g. written by a different/newer converter version).

Common situations: Interrupted parquet converter left a partial file; object store returned an empty/garbage body; version mismatch between the writer's proto schema and the reader's field expectations.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

• read convertedFromBLIDs field from parquet metadata file
• failed to convert matchers
• invalid duration
• bad cached type
• error marshaling proto response

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/eefd02d71a85a37e. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/fetcher.go:1244

func (f *IgnoreParquetConvertedBlocksFilter) readMigratedBlocksFromParquetMetadata(ctx context.Context, path string) (map[ulid.ULID]struct{}, error) {
	migratedBlocks := make(map[ulid.ULID]struct{})
	r, err := f.bkt.Get(ctx, path)
	if err != nil {
		return nil, errors.Wrapf(err, "get parquet metadata file: %v", path)
	}
	defer runutil.CloseWithLogOnErr(f.logger, r, "close bkt get")

	content, err := io.ReadAll(r)
	if err != nil {
		return nil, errors.Wrapf(err, "read parquet metadata file: %v", path)
	}

	var fc easyproto.FieldContext
	for len(content) > 0 {
		content, err = fc.NextField(content)
		if err != nil {
			return nil, errors.Wrapf(err, "read next field from parquet metadata file: %v", path)
		}
		switch fc.FieldNum {
		case 6:
			u, ok := fc.String()
			if !ok {
				return nil, errors.Wrapf(err, "read convertedFromBLIDs field from parquet metadata file: %v", path)
			}
			id, err := ulid.Parse(u)
			if err != nil {
				return nil, errors.Wrapf(err, "parse block ID %q from parquet metadata file: %v", u, path)
			}
			migratedBlocks[id] = struct{}{}
		}
	}

	return migratedBlocks, nil
}

View on GitHub (pinned to 35b8b99117)