argoproj/argo-workflows · error

failed to copy %s to %s: %w

Error message

failed to copy %s to %s: %w

What it means

saveParameter failed while copying the parameter source file's contents to the destination file with io.Copy. The parameter file was opened and created, but the transfer failed mid-way — typically a read error on src or a write/ENOSPC error on dst. The destination file may be partially written and the parameter will be missing or corrupt.

Source

Thrown at cmd/argoexec/commands/emissary.go:756

		return fmt.Errorf("failed to open %s: %w", srcPath, err)
	}
	defer func() { _ = src.Close() }()
	dstPath := varRunArgo + "/outputs/parameters/" + srcPath
	logger.WithFields(logging.Fields{
		"src": srcPath,
		"dst": dstPath,
	}).Info(ctx, "saving parameter")
	z := filepath.Dir(dstPath)
	if mkdirErr := os.MkdirAll(z, 0o755); mkdirErr != nil { // chmod rwxr-xr-x
		return fmt.Errorf("failed to create directory %s: %w", z, mkdirErr)
	}
	dst, err := os.Create(dstPath)
	if err != nil {
		return fmt.Errorf("failed to create %s: %w", srcPath, err)
	}
	defer func() { _ = dst.Close() }()
	if _, err = io.Copy(dst, src); err != nil {
		return fmt.Errorf("failed to copy %s to %s: %w", srcPath, dstPath, err)
	}
	if err = dst.Close(); err != nil {
		return fmt.Errorf("failed to close %s: %w", dstPath, err)
	}
	return nil
}

View on GitHub (pinned to 35bff19146)

Solutions

1. Ensure the script finishes writing the parameter file before the main process exits (write atomically: write temp file then rename).
2. Keep parameter values small — use artifacts for large outputs; free disk space if ENOSPC.
3. Retry the step; persistent errors point to node storage problems — investigate/move workloads.
4. Verify no other process in the pod deletes or truncates the parameter path during execution.

Example fix

// before: in-place writes can be observed mid-write
printf 'partial'; printf ' value' >> /tmp/out
// after: atomic write so the reader never sees a partial file
printf value > /tmp/out.tmp && mv /tmp/out.tmp /tmp/out

// in the step script: write parameters atomically and keep them small
// size check before declaring a parameter from a file
if fi, err := os.Stat(paramPath); err == nil && fi.Size() > 256*1024 {
    return errors.New("parameter too large; use an artifact instead")
}

err := runStep(ctx)
if err != nil && strings.Contains(err.Error(), "failed to copy") {
    // src mutated or ENOSPC mid-copy: retry the step; make scripts write atomically
    return retryWithBackoff(ctx, err)
}

When it happens

Trigger: io.Copy(dst, src) returns error: the source file shrank/was deleted mid-read, src is a special file that errors on read, dst hits ENOSPC during write, or underlying storage I/O error.

Common situations: Script truncates/rotates the output file while the executor reads it; disk fills while writing large parameter values (parameters should be small); failing node storage.

Related errors

• failed to open %s: %w
• failed to create %s: %w
• stat supervisor status: %w
• failed to open stdout: %w
• failed to open combined: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/6f5e96c6fcd75fb3. Report an issue: GitHub.