What causes "failed to parse `df` output; expected at least 4 columns" in hashicorp/nomad?

Ensure GNU/BSD df with -P/-k flags is used (the code already passes -P on linux) so output is single-line.

failed to parse `df` output; expected at least 4 columns

Error message

failed to parse `df` output; expected at least 4 columns

What it means

The data line of `df` output must contain at least 4 whitespace-separated columns (Filesystem, 1024-blocks, Used, Available, ...). If fields < 4, the volume and capacity cannot be extracted and this error is returned.

Source

Thrown at client/fingerprint/storage_unix.go:48

	} else {
		dfArgs = "-k"
	}

	mountOutput, err := exec.Command("df", dfArgs, absPath).Output()
	if err != nil {
		return "", 0, fmt.Errorf("failed to determine mount point for %s", absPath)
	}
	// Output looks something like:
	//	Filesystem 1024-blocks      Used Available Capacity   iused    ifree %iused  Mounted on
	//	/dev/disk1   487385240 423722532  63406708    87% 105994631 15851677   87%   /
	//	[0] volume [1] capacity [2] SKIP  [3] free
	lines := strings.Split(string(mountOutput), "\n")
	if len(lines) < 2 {
		return "", 0, fmt.Errorf("failed to parse `df` output; expected at least 2 lines")
	}
	fields := strings.Fields(lines[1])
	if len(fields) < 4 {
		return "", 0, fmt.Errorf("failed to parse `df` output; expected at least 4 columns")
	}
	volume = fields[0]

	total, err = strconv.ParseUint(fields[1], 10, 64)
	if err != nil {
		return "", 0, fmt.Errorf("failed to parse storage.bytestotal size in kilobytes")
	}
	// convert to bytes
	total *= 1024

	return volume, total, nil
}

View on GitHub (pinned to 482b49bf1a)

Solutions

Ensure GNU/BSD df with -P/-k flags is used (the code already passes -P on linux) so output is single-line.
Check `df -kP <path>` output manually for column count.
Fix locale environment (LC_ALL=C) if a localized df alters output.
Use DiskTotalMB config override to skip df parsing.

Example fix

// before (env)
LC_ALL=de_DE.UTF-8  # localized df output
// after (service env)
LC_ALL=C

Defensive patterns

Strategy: validation

Validate before calling

out, _ := exec.Command("df", "-kP", path).Output()
lines := strings.Split(string(out), "\n")
if len(lines) < 2 || len(strings.Fields(lines[1])) < 4 {
	return fmt.Errorf("df output columns unexpected; normalize df environment (LC_ALL=C, standard df)")
}

Try / catch

if err := fp.Fingerprint(req); err != nil {
	if strings.Contains(err.Error(), "expected at least 4 columns") {
		log.Printf("df columns unexpected: %v", err)
		return nil
	}
	return err
}

Prevention

Run clients with LC_ALL=C to keep df output stable.
Ensure -P (posix) output is available for long paths.
Avoid localized or third-party df implementations.
Provide DiskTotalMB override where df format cannot be controlled.

When it happens

Trigger: strings.Fields(lines[1]) yields fewer than 4 columns because df produced a truncated or differently formatted data row (e.g. wrapped output, localized df, mount names containing newlines).

Common situations: Localized df headers/format changes; long mount paths on df without -P; third-party df replacements with different column layouts.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/eb387513bd8ece01. Report an issue: GitHub.