{"record":{"id":"29e4f815f24c3a50","repo":"kopia/kopia","slug":"new-set","errorCode":null,"errorMessage":"new set","messagePattern":"new set","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"repo/content/content_manager_iterate.go","lineNumber":235,"sourceCode":"\t\t\treturn nil\n\t\t}); err != nil {\n\t\treturn errors.Wrap(err, \"error iterating contents\")\n\t}\n\n\tfor _, v := range packUsage {\n\t\tif err := callback(*v); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// IterateUnreferencedPacks returns the list of unreferenced storage blobs.\nfunc (bm *WriteManager) IterateUnreferencedPacks(ctx context.Context, blobPrefixes []blob.ID, parallelism int, callback func(blob.Metadata) error) error {\n\tusedPacks, err := bigmap.NewSet(ctx)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"new set\")\n\t}\n\n\tdefer usedPacks.Close(ctx)\n\n\tcontentlog.Log(ctx, bm.log, \"determining blobs in use\")\n\t// find packs in use\n\tif err := bm.IteratePacks(\n\t\tctx,\n\t\tIteratePackOptions{\n\t\t\tPrefixes:                           blobPrefixes,\n\t\t\tIncludePacksWithOnlyDeletedContent: true,\n\t\t},\n\t\tfunc(pi PackInfo) error {\n\t\t\tif pi.ContentCount > 0 {\n\t\t\t\tusedPacks.Put(ctx, []byte(pi.PackID))\n\t\t\t}\n\n\t\t\treturn nil","sourceCodeStart":217,"sourceCodeEnd":253,"githubUrl":"https://github.com/kopia/kopia/blob/82495e54b584c1ef6073c9e1be048f57f8aef078/repo/content/content_manager_iterate.go#L217-L253","documentation":"IterateUnreferencedPacks starts by allocating a bigmap.Set to track used pack IDs; if bigmap.NewSet fails, the error is wrapped as 'new set'. bigmap sets spill to temporary storage/memory, so this almost always indicates a failure allocating memory or creating the temp spill files.","triggerScenarios":"Calling WriteManager.IterateUnreferencedPacks when bigmap.NewSet cannot initialize — out of memory, or temp directory unavailable/unwritable.","commonSituations":"Running on memory-constrained hosts or containers with low memory limits, or an environment where the temp directory (TMPDIR) is read-only or full.","solutions":["Free memory or raise the container/host memory limit","Ensure the temp directory used for spilling exists and is writable (check TMPDIR)","Reduce concurrent workloads sharing memory with the iteration","Retry the operation once memory pressure subsides"],"exampleFix":"// before\ncmd := exec.Command(ctx, \"kopia\", \"blob\", \"list\") // inside a 64MB container\n// after\n// raise memory limit for maintenance jobs\ndocker run -m 512m ... kopia maintenance run","handlingStrategy":"validation","validationCode":"if debug.FreeMemory() < 256<<20 { return errors.New(\"insufficient memory for pack set allocation\") }\nif f, err := os.CreateTemp(\"\", \"bigmap-*\"); err != nil { return err } else { f.Close(); os.Remove(f.Name()) }","typeGuard":null,"tryCatchPattern":"usedPacks, err := bigmap.NewSet(ctx)\nif err != nil {\n    return fmt.Errorf(\"cannot allocate pack set (memory/temp dir?): %w\", err)\n}","preventionTips":["Provision adequate memory for maintenance jobs","Ensure TMPDIR is writable with free space","Monitor memory usage during unreferenced-blob scans"],"tags":["go","memory","allocation","bigmap"],"backgroundTag":"resource-not-found","analyzedSha":"82495e54b584c1ef6073c9e1be048f57f8aef078","analyzedAt":"2026-09-07T20:35:21.689Z","contentChangedAt":"2026-09-07T20:35:21.689Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}