karatelabs/karate · error · RuntimeException

Config evaluation failed

Error message

Config evaluation failed: <displayName> - <message>

What it means

evalConfigJs evaluates the JS karate-config.js (or a JS config snippet) to initialize configuration variables. If the script throws or evaluation fails, the code logs a warning and wraps the exception in a RuntimeException naming the config source and the underlying message. The original exception is chained as the cause.

Solutions

  1. Read the trailing part of the message (after 'displayName - ') — it contains the actual JS error and line.
  2. Run the failing statement in isolation in karate-config.js and fix the JS error (undefined var, bad path).
  3. Verify read(...) paths in the config resolve relative to the feature/classpath location.
  4. Add defensive guards in karate-config.js (e.g. karate.env checks) so required inputs exist before use.

Example fix

// before: karate-config.js
var port = java.lang.System.getenv('SERVICE_PORT');
config.serverUrl = 'http://localhost:' + port.trim();

// after
var port = java.lang.System.getenv('SERVICE_PORT');
config.serverUrl = 'http://localhost:' + (port ? port.trim() : '8080');
Defensive patterns

Strategy: validation

Validate before calling

// in karate-config.js, fail fast with clear messages
function fn() {
  var env = karate.env;
  karate.log('karate.env:', env);
  if (!env) throw 'karate.env not set; use -Dkarate.env=...';
  var config = { env: env };
  if (env == 'dev') config.serverUrl = 'http://localhost:8080';
  else throw 'no config for env: ' + env;
  return config;
}

Type guard

function safeEnv(name) {
  var v = java.lang.System.getenv(name);
  if (v == null) throw 'required env var missing: ' + name;
  return v;
}

Try / catch

try {
    ScenarioEngine.get().evalJs(configJs);
} catch (RuntimeException e) {
    if (e.getMessage().startsWith("Config evaluation failed")) {
        throw new IllegalStateException("Check karate-config.js: " + e.getCause(), e);
    }
    throw e;
}

Prevention

When it happens

Trigger: karate-config.js throws a JS runtime error (undefined variable, typo, failed read(), calling karate.* APIs unavailable at config time), or configure() receives JS that fails to evaluate; invoked via evalConfig during ScenarioRuntime startup.

Common situations: Environment-specific config referencing a missing env variable; a read('my-utils.js') inside karate-config.js pointing to a bad path; upgrading Karate and relying on removed config APIs.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of karatelabs/karate@a22eb90246 (2026-09-12). Data as JSON: /api/errors/76ea871784fe980e. Report an issue: GitHub.

Appendix: source

Thrown at karate-core/src/main/java/io/karatelabs/core/ScenarioRuntime.java:504

                // Already evaluated to a value (e.g., object literal)
                result = fn;
            }

            // Apply config variables to engine
            if (result instanceof Map) {
                Map<String, Object> vars = (Map<String, Object>) result;
                for (var entry : vars.entrySet()) {
                    karate.engine.put(entry.getKey(), entry.getValue());
                }
                logger.debug("Evaluated {}: {} variables", displayName, vars.size());
                return vars;
            } else if (result != null) {
                logger.warn("{} did not return an object, got: {}", displayName, result.getClass().getSimpleName());
            }
            return null;
        } catch (Exception e) {
            logger.warn("Failed to evaluate {}: {}", displayName, e.getMessage());
            throw new RuntimeException("Config evaluation failed: " + displayName + " - " + e.getMessage(), e);
        }
    }

    /**
     * Execute the @setup scenario and return all its variables.
     */
    public Map<String, Object> executeSetup(String name) {
        if (featureRuntime == null) {
            throw new RuntimeException("karate.setup() requires a feature context");
        }
        Scenario setupScenario = scenario.getFeature().getSetup(name);
        if (setupScenario == null) {
            String message = "no scenario found with @setup tag";
            if (name != null) {
                message = message + " and name '" + name + "'";
            }
            throw new RuntimeException(message);
        }

View on GitHub (pinned to a22eb90246)