{"record":{"id":"94d49f5c8c48c6b6","repo":"thanos-io/thanos","slug":"write-meta","errorCode":null,"errorMessage":"write meta","messagePattern":"write meta","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/thanos/downsample.go","lineNumber":416,"sourceCode":"\t\terr = stats.AnyErr()\n\t}\n\tif err != nil && !acceptMalformedIndex {\n\t\treturn errors.Wrap(err, \"output block index not valid\")\n\t}\n\n\tmeta, err := metadata.ReadFromDir(resdir)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"read meta\")\n\t}\n\n\tif stats.ChunkMaxSize > 0 {\n\t\tmeta.Thanos.IndexStats.ChunkMaxSize = stats.ChunkMaxSize\n\t}\n\tif stats.SeriesMaxSize > 0 {\n\t\tmeta.Thanos.IndexStats.SeriesMaxSize = stats.SeriesMaxSize\n\t}\n\tif err := meta.WriteToDir(logger, resdir); err != nil {\n\t\treturn errors.Wrap(err, \"write meta\")\n\t}\n\n\tbegin = time.Now()\n\n\terr = block.Upload(ctx, logger, bkt, resdir, hashFunc, objstore.WithUploadConcurrency(blockFilesConcurrency))\n\tif err != nil {\n\t\treturn compact.NewRetryError(errors.Wrapf(err, \"upload downsampled block %s\", id))\n\t}\n\n\tlevel.Info(logger).Log(\"msg\", \"uploaded block\", \"id\", id, \"duration\", time.Since(begin), \"duration_ms\", time.Since(begin).Milliseconds())\n\n\t// It is not harmful if these fails.\n\tif err := os.RemoveAll(bdir); err != nil {\n\t\tlevel.Warn(logger).Log(\"msg\", \"failed to clean directory\", \"dir\", bdir, \"err\", err)\n\t}\n\tif err := os.RemoveAll(resdir); err != nil {\n\t\tlevel.Warn(logger).Log(\"msg\", \"failed to clean directory\", \"resdir\", bdir, \"err\", err)\n\t}","sourceCodeStart":398,"sourceCodeEnd":434,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/cmd/thanos/downsample.go#L398-L434","documentation":"processDownsampling wraps meta.WriteToDir failures as \"write meta\". The enriched metadata (with index stats chunk/series max sizes) cannot be written to the output block's meta.json. This blocks the subsequent upload of the downsampled block.","triggerScenarios":"meta.WriteToDir(logger, resdir) fails: permission denied on resdir, disk full, resdir removed mid-run, or filesystem I/O error while serializing meta.json.","commonSituations":"--data-dir on a full or read-only volume; quota exceeded on ephemeral storage in Kubernetes; concurrent processes racing over the same data dir; SELinux/AppArmor blocking writes.","solutions":["Verify --data-dir has free space and is writable by the thanos process user; clean stale block dirs.","In Kubernetes, size emptyDir/PV adequately and check kubelet eviction logs.","Ensure a single downsampler instance owns the data dir; remove MAC (SELinux) denials from audit logs if present.","Re-run after cleanup — the operation is idempotent per block and will regenerate the output."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// preflight before writing\nif err := syscall.Access(resdir, os.O_WRONLY); err != nil {\n    return fmt.Errorf(\"output dir %s not writable: %w\", resdir, err)\n}","typeGuard":null,"tryCatchPattern":"if err := meta.WriteToDir(logger, resdir); err != nil {\n    return errors.Wrap(err, \"write meta\") // check disk space/permissions before rerunning\n}","preventionTips":["Alert on data-dir free space before it hits zero.","Run the downsampler with a dedicated user owning the data dir; avoid restrictive MAC policies.","Use persistent, correctly-sized storage for --data-dir in containerized deployments."],"tags":["metadata","filesystem","write"],"backgroundTag":"file-write-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}