apple/pkl · error · VmException

invalidOutputFileStructure

invalidOutputFileStructure

Error message

invalidOutputFileStructure

What it means

While validating example outputs, `doRunAndValidateExamples` re-loads the `*-actual.pcf` file it wrote earlier in the same run. If a group or example member that was just written is missing from that file, the file must have been modified or replaced by another process between the write and the read, so the runner throws `invalidOutputFileStructure` naming the actual-output file.

Source

Thrown at pkl-core/src/main/java/org/pkl/core/runtime/TestRunner.java:254

                  if (actualExampleOutputs.isNull()) {
                    // immediately write and load `<fileName>-actual.pcf`
                    // so that we can generate deep link with correct line number for each
                    // mismatch
                    writeExampleOutputs(actualOutputFile, examples);
                    actualExampleOutputs.set(loadExampleOutputs(actualOutputFile));
                  }

                  var expectedMember = VmUtils.findMember(expectedGroup, exampleIndex);
                  assert expectedMember != null;

                  var actualGroup =
                      (VmObjectLike) VmUtils.readMemberOrNull(actualExampleOutputs.get(), groupKey);
                  var actualMember =
                      actualGroup == null ? null : VmUtils.findMember(actualGroup, exampleIndex);
                  if (actualMember == null) {
                    // file was written earlier in this method;
                    // must have been tampered with by another process
                    throw new VmExceptionBuilder()
                        .evalError("invalidOutputFileStructure", actualOutputFile)
                        .build();
                  }

                  testResultBuilder.addFailure(
                      exampleFailure(
                          getDisplayUri(exampleMember),
                          getDisplayUri(expectedMember),
                          expectedValuePcf,
                          getDisplayUri(actualMember),
                          exampleValuePcf,
                          testResultBuilder.getCount()));
                } else {
                  testResultBuilder.addSuccess();
                }

                return true;
              }));

View on GitHub (pinned to f3efcbfc9b)

Solutions

1. Stop concurrent processes (formatters, watchers, parallel runs) from touching `*-actual.pcf` during the test run.
2. Ensure only one `pkl test` process runs against the same module/directory at a time.
3. Rerun the test on a clean checkout to rule out a transient clobber.
4. Exclude generated `*-actual.pcf`/`*-expected.pcf` files from auto-format/autosave tooling.

Example fix

// before: two parallel jobs test the same module in one shared dir
// after: give each job its own copy
mkdir -p "$RUNNER_TEMP/test-${CI_JOB_ID}"
cp -r . "$RUNNER_TEMP/test-${CI_JOB_ID}" && pkl test MyTest.pkl # inside the isolated copy

catch (EvalError e) {
  if (e.getMessage().contains("invalidOutputFileStructure")) {
    // rerun test; if it recurs, hunt for concurrent writers
  }
}

When it happens

Trigger: Another process edits, truncates, locks, or deletes the `*-actual.pcf` file between the write and the re-read during a `pkl test` run; a file watcher or formatter rewrites the file mid-test.

Common situations: An auto-formatter or linter hook reformatting generated Pcf files on save; parallel test runs sharing one directory and clobbering each other's actual-output files; sync clients reverting files.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• cannotEvaluateNonFileBasedTestModule
• ioErrorWritingTestOutputFile
• ioErrorReadingTestOutputFile
• resourcePastRootDir|modulePastRootDir
• Is a directory

AI-assisted analysis of apple/pkl@f3efcbfc9b (2026-09-08). Data as JSON: /api/errors/2895898235e9f41a. Report an issue: GitHub.