{"record":{"id":"41def34f3239c9da","repo":"vxcontrol/pentagi","slug":"tar-archive-exceeds-maximum-file-count-of-d","errorCode":null,"errorMessage":"tar archive exceeds maximum file count of %d","messagePattern":"tar archive exceeds maximum file count of (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"backend/pkg/flowfiles/files.go","lineNumber":407,"sourceCode":"\t\t}\n\n\t\tentryPath := filepath.Join(destDir, filepath.Clean(filepath.FromSlash(hdr.Name)))\n\t\tif !IsWithinDir(entryPath, destDir) {\n\t\t\tcontinue\n\t\t}\n\n\t\tswitch hdr.Typeflag {\n\t\tcase tar.TypeDir:\n\t\t\tif err := os.MkdirAll(entryPath, 0755); err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to create directory '%s': %w\", entryPath, err)\n\t\t\t}\n\t\tcase tar.TypeReg, tar.TypeRegA:\n\t\t\tif hdr.Size < 0 {\n\t\t\t\treturn fmt.Errorf(\"tar entry '%s' has invalid size %d\", hdr.Name, hdr.Size)\n\t\t\t}\n\t\t\tfilesCount++\n\t\t\tif filesCount > MaxPullFiles {\n\t\t\t\treturn fmt.Errorf(\"tar archive exceeds maximum file count of %d\", MaxPullFiles)\n\t\t\t}\n\t\t\ttotalSize += hdr.Size\n\t\t\tif totalSize > MaxPullTotalSize {\n\t\t\t\treturn fmt.Errorf(\"tar archive exceeds maximum total size of %d bytes\", MaxPullTotalSize)\n\t\t\t}\n\n\t\t\tif err := os.MkdirAll(filepath.Dir(entryPath), 0755); err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to create parent directory for '%s': %w\", entryPath, err)\n\t\t\t}\n\n\t\t\tf, err := os.OpenFile(entryPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0644)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to create file '%s': %w\", entryPath, err)\n\t\t\t}\n\t\t\t_, copyErr := io.CopyN(f, tr, hdr.Size)\n\t\t\tf.Close()\n\t\t\tif copyErr != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to write file '%s': %w\", entryPath, copyErr)","sourceCodeStart":389,"sourceCodeEnd":425,"githubUrl":"https://github.com/vxcontrol/pentagi/blob/ea665308baaff015b226f308438a68d929d0f29b/backend/pkg/flowfiles/files.go#L389-L425","documentation":"ExtractTar enforces MaxPullFiles (1000) regular-file entries per archive; exceeding it aborts extraction with this error. It is a deliberate anti-DoS limit so a small malicious tarball cannot exhaust inodes or disk via many files.","triggerScenarios":"Calling ExtractTar (directly or via PullFlowFiles) on an archive containing more than 1000 regular-file entries; file count is checked incrementally as entries stream through.","commonSituations":"Backing up a dependency tree (node_modules-like), a dataset directory with thousands of small files, or recursively including uploads; users legitimately archiving large project trees then pulling them into a flow.","solutions":["Split the archive into chunks of ≤1000 files and pull them separately.","Exclude unnecessary paths (dependencies, caches, .git) when creating the tar on the source side.","If the workload legitimately needs more files, raise the MaxPullFiles constant in backend/pkg/flowfiles/files.go and redeploy — accepting the higher resource cost.","Tar a single compressed bundle (one big file) instead of many small files."],"exampleFix":"// before\ntar -cf bundle.tar ./project/\n// after\ntar -cf bundle.tar --exclude='./project/.git' --exclude='./project/node_modules' ./project/","handlingStrategy":"validation","validationCode":"// Count regular files without extracting (Go or shell):\n// tar -tf bundle.tar | grep -vc '/$'\n// if the count exceeds 1000, split or prune before calling ExtractTar.","typeGuard":null,"tryCatchPattern":"err := flowfiles.ExtractTar(rc, destDir)\nif err != nil {\n    if strings.Contains(err.Error(), \"exceeds maximum file count\") {\n        return fmt.Errorf(\"archive has too many files (limit %d); split or exclude paths\", flowfiles.MaxPullFiles)\n    }\n    return err\n}","preventionTips":["Exclude dependency/cache directories (.git, node_modules) when creating flow archives.","Count archive members with `tar -tf | wc -l` before pulling large project trees.","Prefer single-file compressed bundles over many small files."],"tags":["tar","limits","security","dos-protection"],"backgroundTag":"file-count-limit-exceeded","analyzedSha":"ea665308baaff015b226f308438a68d929d0f29b","analyzedAt":"2026-09-01T14:16:31.421Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}