What causes "Failed to set permissions to {}: {e}" in windmill-labs/windmill?

Check the wrapped io::Error detail in the message (EPERM/EACCES/EINVAL etc.) to identify the cause.

Failed to set permissions to {}: {e}

Error message

Failed to set permissions to {}: {e}

What it means

Raised in write_file_at_user_defined_location when File::set_permissions fails after creating the file at the user-defined path inside the job directory. The unix branch converts the requested numeric mode via PermissionsExt::from_mode and applies it; an OS-level chmod failure (e.g. EPERM) produces this error, wrapping the original io::Error message with the path.

Source

Thrown at backend/windmill-common/src/worker.rs:958

    job_dir: &str,
    user_defined_path: &str,
    content: &str,
    mode: Option<u32>,
) -> error::Result<PathBuf> {
    let normalized_full_path = is_allowed_file_location(job_dir, user_defined_path)?;

    let full_path = normalized_full_path.as_path();
    if let Some(parent_dir) = full_path.parent() {
        std::fs::create_dir_all(parent_dir)?;
    }

    let mut file = File::create(full_path)?;

    #[cfg(unix)]
    if let Some(mode) = mode {
        let perm = std::os::unix::fs::PermissionsExt::from_mode(mode);
        file.set_permissions(perm)
            .map_err(|e| anyhow!("Failed to set permissions to {}: {e}", user_defined_path))?;
    }

    #[cfg(windows)]
    if mode.is_some() {
        tracing::error!("Cannot use `mode` to set file permissions on windows workers");
    }

    file.write_all(content.as_bytes())?;
    file.flush()?;
    Ok(normalized_full_path)
}

pub async fn reload_custom_tags_setting(db: &DB) -> error::Result<()> {
    let q =
        crate::global_settings::load_value_from_global_settings(db, CUSTOM_TAGS_SETTING).await?;
    apply_custom_tags_setting(q);
    Ok(())
}

View on GitHub (pinned to e474e8803c)

Solutions

Check the wrapped io::Error detail in the message (EPERM/EACCES/EINVAL etc.) to identify the cause.
Confirm the job directory filesystem supports unix permission bits (avoid SMB/CIFS or restricted NFS mounts for the worker run dir).
Verify no SELinux/AppArmor policy denies chmod in the worker container/host.
Drop or adjust the `mode` argument so the file keeps default permissions if the environment cannot honor it.

Example fix

// before
let path = write_file_at_user_defined_location(job_dir, "run.sh", content, Some(0o755))?;
// after
let path = match write_file_at_user_defined_location(job_dir, "run.sh", content, Some(0o755)) {
    Ok(p) => p,
    Err(e) if e.to_string().contains("Failed to set permissions") => {
        tracing::warn!("chmod unsupported on this volume; writing without explicit mode");
        write_file_at_user_defined_location(job_dir, "run.sh", content, None)?
    }
    Err(e) => return Err(e),
};

Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-check the target volume supports chmod before requesting a mode
use std::os::unix::fs::PermissionsExt;
let probe = std::path::Path::new(job_dir).join(".perm_probe");
std::fs::write(&probe, b"")?;
let ok = std::fs::set_permissions(&probe, std::fs::Permissions::from_mode(0o600)).is_ok();
let _ = std::fs::remove_file(&probe);
if !ok && mode.is_some() {
    eprintln!("filesystem does not honor chmod; mode will be ignored");
}

Try / catch

match write_file_at_user_defined_location(job_dir, path, content, Some(0o755)) {
    Ok(p) => p,
    Err(e) if e.to_string().contains("Failed to set permissions") => {
        // chmod unsupported on this volume; fall back to default perms
        write_file_at_user_defined_location(job_dir, path, content, None)?
    }
    Err(e) => return Err(e.into()),
}

Prevention

Place worker job directories on filesystems that support unix permission bits (ext4/xfs, not SMB/CIFS).
Check container/SELinux/AppArmor policies permit chmod in the job directory.
Avoid mounting the job dir read-only or with mode-stripping options.
Only pass an explicit `mode` when the environment actually needs it; default perms suffice in most cases.
Catch the "Failed to set permissions" prefix and degrade gracefully rather than failing the whole job.

When it happens

Trigger: A job writes a file (via create_file_resources) with an explicit `mode` (permission bits) on a unix worker, and chmod on the just-created file fails: filesystem doesn't support permissions, immutable file/dir, ACL/SELinux denial, or ownership quirks on mounted volumes.

Common situations: Job directory on a filesystem without unix permission support (some NFS mounts, certain overlay/container volumes, CIFS/SMB shares); AppArmor/SELinux policy blocking chmod; read-only remount; running in a container where the mount strips capability to change modes.

Related errors

AI-assisted analysis of windmill-labs/windmill@e474e8803c (2026-09-03). Data as JSON: /api/errors/327ab2bc16569e6b. Report an issue: GitHub.