What causes "builder '%s' run failed to start" in hyperledger/fabric?

Verify bin/run exists and is executable in the builder path

builder '%s' run failed to start

Error message

builder '%s' run failed to start

What it means

Returned by externalbuilder Builder.Run when the builder's bin/run process fails to launch (Start error). The launch dir and chaincode.json were prepared, but the external builder executable could not even be started, so session creation failed.

Source

Thrown at core/container/externalbuilder/externalbuilder.go:383

		return nil, errors.WithMessage(err, "could not create temp run dir")
	}

	rc := newRunConfig(ccid, peerConnection, b.MSPID)
	marshaledRC, err := json.Marshal(rc)
	if err != nil {
		return nil, errors.WithMessage(err, "could not marshal run config")
	}

	if err := os.WriteFile(filepath.Join(launchDir, "chaincode.json"), marshaledRC, 0o600); err != nil {
		return nil, errors.WithMessage(err, "could not write root cert")
	}

	run := filepath.Join(b.Location, "bin", "run")
	cmd := b.NewCommand(run, bldDir, launchDir)
	sess, err := Start(b.Logger, cmd, func(error) { os.RemoveAll(launchDir) })
	if err != nil {
		os.RemoveAll(launchDir)
		return nil, errors.Wrapf(err, "builder '%s' run failed to start", b.Name)
	}

	return sess, nil
}

// runCommand runs a command and waits for it to complete.
func (b *Builder) runCommand(cmd *exec.Cmd) error {
	sess, err := Start(b.Logger, cmd)
	if err != nil {
		return err
	}
	return sess.Wait()
}

// NewCommand creates an exec.Cmd that is configured to prune the calling
// environment down to the environment variables specified in the external
// builder's PropagateEnvironment and the DefaultPropagateEnvironment.
func (b *Builder) NewCommand(name string, args ...string) *exec.Cmd {

View on GitHub (pinned to 2736b63f8f)

Solutions

Verify bin/run exists and is executable in the builder path
Check the builder 'path' in externalBuilders config points to the correct location
Inspect the wrapped exec error (shebang, ENOENT, resource limits) and fix accordingly

Example fix

// before (builder dir)
bin/build  bin/release
// after
bin/build  bin/release  bin/run   # chmod +x bin/run

Defensive patterns

Strategy: validation

Validate before calling

runBin := filepath.Join(builderPath, "bin", "run")
if info, err := os.Stat(runBin); err != nil || info.Mode()&0o111 == 0 {
    return fmt.Errorf("builder %s missing executable bin/run", builderPath)
}
out, err := exec.Command(runBin, "--help").CombinedOutput() // optional smoke test
_ = out; _ = err

Try / catch

sess, err := builder.Run(bldDir, launchDir)
if err != nil {
    log.Errorf("run failed to start: %v — check bin/run exists, is executable, and host fd/proc limits", err)
}

Prevention

Ship bin/run executable in every builder image/release
Validate the builder path in core.yaml exists on the host
Check host ulimits (nproc, nofile) if launches fail under load

When it happens

Trigger: Builder.Run() creates a launchDir, runs <builder>/bin/run <bldDir> <launchDir> via Start; exec fails immediately — binary missing, not executable, fork/exec resource error.

Common situations: bin/run missing or lacking executable bit; builder path wrong in core.yaml; too many processes/file descriptors on the host; bad interpreter shebang in the run script.

Related errors

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/5119e535be894373. Report an issue: GitHub.