apple/pkl · error · CliTestException

testsFailed

testsFailed

Error message

ErrorMessages.create("testsFailed")

What it means

After running all test modules, if any test failed, evalTest throws CliTestException with the "testsFailed" message and a non-zero exit code (1, or 10 when an example-failure snapshot was written). This is the CLI's normal signal that the test suite did not pass; it is not a malfunction of the tool itself.

Source

Thrown at pkl-cli/src/main/kotlin/org/pkl/cli/CliTestRunner.kt:137

          }
          if (moduleUri != sources.last()) {
            errWriter.appendLine()
          }
          errWriter.flush()
          failed = true
        }
      }
      if (testOptions.junitAggregateReports && junitDir != null) {
        val fileName = "${testOptions.junitAggregateSuiteName}.xml"
        JUnitReporter(testOptions.junitAggregateSuiteName)
          .summarizeToPath(allTestResults, junitDir.resolve(fileName))
      }
      consoleWriter.append('\n')
      reporter.summarize(allTestResults, consoleWriter)
      consoleWriter.flush()
      if (failed) {
        val exitCode = if (isExampleWrittenFailure) 10 else 1
        throw CliTestException(ErrorMessages.create("testsFailed"), exitCode)
      }
    }
  }
}

View on GitHub (pinned to f3efcbfc9b)

Solutions

1. Read the test summary output to find failing tests and update the code or expected values accordingly.
2. If output changes are intentional, regenerate/update the expected example files (exit code 10 indicates example-written failures where new output was written for inspection).
3. Re-run `pkl test` locally before pushing to confirm all tests pass.

Example fix

// before: expected output in test
output { text = "v1" }

// after intentional change: update the expectation
output { text = "v2" }

// run pkl test and branch on exit codes (1 = failures, 10 = example output written)
val proc = ProcessBuilder("pkl", "test", *modules).start()
val code = proc.waitFor()
when (code) {
  0 -> "" // all passed
  10 -> println("example outputs were rewritten; review and commit them")
  else -> error("pkl test failed with exit code $code")
}

When it happens

Trigger: Running `pkl test <modules>` where at least one test assertion fails; the summary reporter reports failures and `failed` is true at the end of evalTest.

Common situations: A code change broke expected Pkl values rendered in tests; expected example output is stale after intentional changes; CI run picks up a failing test authored earlier.

Related errors

• packageTestsFailed
• Usage: pkl test [<options>] <modules>... Error: missing arg
• Cannot generate JUnit report for $moduleUri. A report with t
• invalidUsageOfProjectFromNonFileUri
• +e.getMessage()

AI-assisted analysis of apple/pkl@f3efcbfc9b (2026-09-08). Data as JSON: /api/errors/dc2da25b41830bac. Report an issue: GitHub.