pathwaycom/pathway · error · KeyError

variable {v} is not defined

Error message

variable {v} is not defined

What it means

KeyError raised by the YAML variable resolver when a $variable used in a Pathway YAML file is found neither in the provided variables mapping (nor its parent resolvers) nor in the environment. For all-uppercase names (optionally with underscores) the resolver falls back to os.environ before failing.

Source

Thrown at python/pathway/internals/yaml_loader.py:161

        if self.parent is not None:
            return self.parent.resolve_variable(v)

        if all(c.upper() or c == "_" for c in v.name):
            s = os.environ.get(v.name)
            if s is not None:
                # using yaml.Loader instead of PathwayYamlLoader to prevent recursive
                # parsing of environment variables
                parsed_value = yaml.load(s, yaml.Loader)
                if (
                    isinstance(parsed_value, int)
                    or isinstance(parsed_value, float)
                    or isinstance(parsed_value, bool)
                ):
                    return parsed_value
                else:
                    return s

        raise KeyError(f"variable {v} is not defined")

    def resolve_variable(self, v: Variable) -> object:
        res = self._resolve_variable(v)
        self.context[v] = res
        self.done[id(res)] = res
        return res

    def __enter__(self) -> Self:
        return self

    def __exit__(
        self,
        exc_type: type[BaseException] | None,
        exc_value: BaseException | None,
        traceback: TracebackType | None,
        /,
    ) -> None:
        if exc_value is not None:

View on GitHub (pinned to fa2f74a464)

Solutions

1. Pass the missing variable: pw.load_yaml_from_file('pipeline.yaml', variables={'password': ...})
2. For env-sourced variables, use an ALL_UPPERCASE name (letters, digits and _ only) and export it in the environment
3. Audit the YAML for all $ occurrences and ensure each is defined

Example fix

# pipeline.yaml uses $API_KEY
# before
pw.load_yaml_from_file('pipeline.yaml', variables={})

# after
pw.load_yaml_from_file('pipeline.yaml', variables={'API_KEY': os.environ['API_KEY']})

import re

def yaml_variables_defined(text: str, variables: dict) -> list[str]:
    used = set(re.findall(r'\$(\w+)', text))
    provided = set(variables)
    return sorted(used - provided)

try:
    pw.load_yaml_from_file('pipeline.yaml', variables=variables)
except KeyError as e:
    msg = str(e)
    if 'variable' in msg and 'is not defined' in msg:
        missing = msg.split('"')[1] if '"' in msg else msg
        raise RuntimeError(f'Missing YAML variable: {missing}; pass it via variables= or env') from e
    raise

When it happens

Trigger: YAML contains $password, $api_key or ${PORT} but pw.load_yaml_from_file(..., variables={...}) does not define it and (for all-caps names) no matching env var exists. Mixed/lowercase names never consult the environment, so they must be passed in the variables dict.

Common situations: Deploying a YAML pipeline without its secrets env vars; renaming a variable in YAML but not in the variables dict; assuming lowercase variables are read from the environment (only ALL_CAPS names are); CI missing env configuration.

Related errors

• expected string key, got {type(key)}
• Failed to install dependencies
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• Column {api.DIFF_PSEUDOCOLUMN} can only have 1 and -1 values
• schema does not match given dataframe

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/e68eea97f3573a60. Report an issue: GitHub.