{"record":{"id":"92a11f504a4ce971","repo":"argoproj/argo-workflows","slug":"failed-to-write-large-arg-d-to-file-w","errorCode":null,"errorMessage":"failed to write large arg %d to file: %w","messagePattern":"failed to write large arg (.+?) to file: %w","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/argoexec/commands/emissary.go","lineNumber":122,"sourceCode":"\t\targsData, readErr := os.ReadFile(argsFile)\n\t\tif readErr != nil {\n\t\t\treturn fmt.Errorf(\"failed to read container args file %s: %w\", argsFile, readErr)\n\t\t}\n\t\tvar fileArgs []string\n\t\tif err = json.Unmarshal(argsData, &fileArgs); err != nil {\n\t\t\treturn fmt.Errorf(\"failed to unmarshal container args: %w\", err)\n\t\t}\n\t\targs = append(args, fileArgs...)\n\t\tlogger.WithField(\"count\", len(fileArgs)).Info(ctx, \"Loaded container args from file\")\n\n\t\t// Check for a large args and offload to file if needed\n\t\t// This avoids the exec() \"argument list too long\" error\n\t\t// Downstream programs should support @filename for parsing large args\n\t\tfor i := 0; i < len(args); i++ {\n\t\t\tif len(args[i]) > common.MaxEnvVarLen {\n\t\t\t\tfilePath := fmt.Sprintf(\"/tmp/argo_arg_%d.txt\", i)\n\t\t\t\tif err = os.WriteFile(filePath, []byte(args[i]), 0o644); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"failed to write large arg %d to file: %w\", i, err)\n\t\t\t\t}\n\t\t\t\tlogger.WithFields(logging.Fields{\n\t\t\t\t\t\"argIndex\": i,\n\t\t\t\t\t\"size\":     len(args[i]),\n\t\t\t\t\t\"filePath\": filePath,\n\t\t\t\t}).Info(ctx, \"Offloaded large argument to file. Downstream program must support @filename syntax\")\n\t\t\t\targs[i] = \"@\" + filePath\n\t\t\t}\n\t\t}\n\t}\n\n\t// In init-less pod mode the supervisor, not an init container, writes\n\t// /var/run/argo/template. Supervisor and main start concurrently, so\n\t// block until supervisor signals readiness (or failure) before reading\n\t// the template. Gated on an env var so legacy pods are unaffected.\n\twaitForReady := os.Getenv(common.EnvVarWaitForReady) == \"true\"\n\tif waitForReady {\n\t\tif waitErr := waitForSupervisorReady(ctx); waitErr != nil {","sourceCodeStart":104,"sourceCodeEnd":140,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/cmd/argoexec/commands/emissary.go#L104-L140","documentation":"After loading args, emissary checks each arg: any single argument longer than common.MaxEnvVarLen is written to /tmp/argo_arg_<i>.txt and replaced in argv with the literal `@<path>` (the downstream program must support @filename syntax). This error is thrown when os.WriteFile to that temp path fails — typically no space, a read-only /tmp, or a permission issue — so the oversized arg cannot be offloaded and exec would otherwise blow the argument-size limit.","triggerScenarios":"/tmp inside the workflow container is full or a read-only mount; the container user lacks write permission on /tmp; an existing /tmp/argo_arg_<i>.txt is owned by another user and unwritable; diskPressure evictions/emptyDir quota exhausted.","commonSituations":"Workflows with huge script bodies or giant parameters passed as container args; containers running as a non-root user with a restrictive securityContext; pods mounting an emptyDir over /tmp with a small sizeLimit; nested tooling that also writes /tmp/argo_arg_*.txt and races the emissary.","solutions":["Free space / raise the sizeLimit of the volume mounted at /tmp (emptyDir sizeLimit or the node's disk).","Ensure the container's securityContext allows writing /tmp (run as a user with write access; /tmp mode 1777).","Reduce the size of the offending argument — e.g. pass it via an artifact or mounted file instead of as a container arg.","Have the downstream program consume the @filename indirection (emissary replaces the arg with `@/tmp/argo_arg_<i>.txt`); if it does not, restructure the workflow.","Remove stale /tmp/argo_arg_*.txt files from the image or an init step if they could collide."],"exampleFix":"// before: pod with read-only small /tmp\nvolumes:\n  - name: tmp\n    emptyDir: { sizeLimit: 1Mi }\ncontainers:\n  - securityContext: { readOnlyRootFilesystem: true }\n// after: writable, adequately sized /tmp\nvolumes:\n  - name: tmp\n    emptyDir: { sizeLimit: 1Gi }\ncontainers:\n  - volumeMounts: [ { name: tmp, mountPath: /tmp } ]","handlingStrategy":"validation","validationCode":"// Pre-check writability and space of the offload dir before emissary runs:\nfunc validateTmpWritable(dir string) error {\n\tprobe := filepath.Join(dir, \".argo-write-probe\")\n\tif err := os.WriteFile(probe, []byte(\"ok\"), 0o644); err != nil {\n\t\treturn fmt.Errorf(\"%s not writable: %w\", dir, err)\n\t}\n\treturn os.Remove(probe)\n}","typeGuard":null,"tryCatchPattern":"if err := os.WriteFile(filePath, []byte(args[i]), 0o644); err != nil {\n\tswitch {\n\tcase errors.Is(err, os.ErrPermission):\n\t\tlog.Errorf(\"cannot write arg offload file (permissions/readonly fs): %v\", err)\n\tcase errors.Is(err, syscall.ENOSPC):\n\t\tlog.Errorf(\"no space left for arg offload file: %v\", err)\n\t}\n\treturn err\n}","preventionTips":["Mount a writable emptyDir at /tmp with a sizeLimit sized to your largest argument.","Avoid readOnlyRootFilesystem without a writable /tmp volume.","Run containers as a user with write access to /tmp (mode 1777).","Keep individual arguments under MaxEnvVarLen; pass huge payloads as artifacts/files instead.","Ensure the downstream program supports @filename argument syntax."],"tags":["filesystem","executor","disk","permissions","argo-workflows"],"backgroundTag":"temp-file-write-failed","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}