windmill-labs/windmill · error

File ${path} is encoded as UTF-16 LE, which is not supported

Error message

File ${path} is encoded as UTF-16 LE, which is not supported. Please convert it to UTF-8.

What it means

Sentinel guard in decodeBufferAsUtf8: the file read starts with the UTF-16 LE BOM (FF FE), so it is not valid UTF-8 and decoding it as UTF-8 would silently corrupt every string parsed from it (scripts, resources, variables). The input at fault is the file's encoding.

Source

Thrown at cli/src/utils/utils.ts:143

  return await generateHashFromBuffer(messageBuffer);
}

export async function generateHashFromBuffer(
  content: BufferSource
): Promise<string> {
  const hashBuffer = await crypto.subtle.digest("SHA-256", content);
  return Buffer.from(hashBuffer).toString("hex");
}

function decodeBufferAsUtf8(buf: Buffer, path: string | URL): string {
  if (buf.length >= 2) {
    if (buf[0] === 0xff && buf[1] === 0xfe) {
      if (buf.length >= 4 && buf[2] === 0x00 && buf[3] === 0x00) {
        throw new Error(
          `File ${path} is encoded as UTF-32 LE, which is not supported. Please convert it to UTF-8.`
        );
      }
      throw new Error(
        `File ${path} is encoded as UTF-16 LE, which is not supported. Please convert it to UTF-8.`
      );
    }
    if (buf[0] === 0xfe && buf[1] === 0xff) {
      throw new Error(
        `File ${path} is encoded as UTF-16 BE, which is not supported. Please convert it to UTF-8.`
      );
    }
    if (buf.length >= 4 && buf[0] === 0x00 && buf[1] === 0x00 && buf[2] === 0xfe && buf[3] === 0xff) {
      throw new Error(
        `File ${path} is encoded as UTF-32 BE, which is not supported. Please convert it to UTF-8.`
      );
    }
  }
  if (buf.length >= 3 && buf[0] === 0xef && buf[1] === 0xbb && buf[2] === 0xbf) {
    return buf.subarray(3).toString("utf-8");
  }
  return buf.toString("utf-8");

View on GitHub (pinned to e474e8803c)

Solutions

  1. Re-save the file as UTF-8 in your editor
  2. Convert: `iconv -f UTF-16LE -t UTF-8 file > file.tmp && mv file.tmp file`
  3. In PowerShell 5.1, pass `-Encoding utf8` to Out-File/Set-Content instead of the default

Example fix

// before (PowerShell 5.1)
Get-Content raw.yaml | Set-Content raw.yaml  # UTF-16 LE
// after
Get-Content raw.yaml | Set-Content raw.yaml -Encoding utf8
Defensive patterns

Strategy: validation

Validate before calling

const fs = require('fs');
function assertNotUtf16Le(path) {
  const b = fs.readFileSync(path).subarray(0, 4);
  if (b[0] === 0xff && b[1] === 0xfe && !(b[2] === 0x00 && b[3] === 0x00))
    throw new Error(`${path} is UTF-16 LE; convert to UTF-8 first`);
}

Type guard

function isUtf16Le(buf) {
  return buf.length >= 2 && buf[0] === 0xff && buf[1] === 0xfe && !(buf.length >= 4 && buf[2] === 0x00 && buf[3] === 0x00);
}

Try / catch

try {
  const content = await readTextFile(path);
} catch (e) {
  if (String(e.message).includes('UTF-16 LE')) console.error('Convert with: iconv -f UTF-16LE -t UTF-8 <file>');
  else throw e;
}

Prevention

When it happens

Trigger: Reading a file via `readTextFile`/`readTextFileSync` whose first two bytes are FF FE (and not followed by 00 00).

Common situations: Files exported from Windows Notepad ('Unicode' encoding), Excel CSV exports, or PowerShell 5.1 `Out-File` default (UTF-16 LE).

Related errors


AI-assisted analysis of windmill-labs/windmill@e474e8803c (2026-09-03). Data as JSON: /api/errors/b7fe90767643a917. Report an issue: GitHub.