{"record":{"id":"dc2da25b41830bac","repo":"apple/pkl","slug":"testsfailed","errorCode":"testsFailed","errorMessage":"ErrorMessages.create(\"testsFailed\")","messagePattern":"ErrorMessages\\.create\\(\"testsFailed\"\\)","errorType":"error_code","errorClass":"CliTestException","httpStatus":null,"severity":"error","filePath":"pkl-cli/src/main/kotlin/org/pkl/cli/CliTestRunner.kt","lineNumber":137,"sourceCode":"          }\n          if (moduleUri != sources.last()) {\n            errWriter.appendLine()\n          }\n          errWriter.flush()\n          failed = true\n        }\n      }\n      if (testOptions.junitAggregateReports && junitDir != null) {\n        val fileName = \"${testOptions.junitAggregateSuiteName}.xml\"\n        JUnitReporter(testOptions.junitAggregateSuiteName)\n          .summarizeToPath(allTestResults, junitDir.resolve(fileName))\n      }\n      consoleWriter.append('\\n')\n      reporter.summarize(allTestResults, consoleWriter)\n      consoleWriter.flush()\n      if (failed) {\n        val exitCode = if (isExampleWrittenFailure) 10 else 1\n        throw CliTestException(ErrorMessages.create(\"testsFailed\"), exitCode)\n      }\n    }\n  }\n}\n","sourceCodeStart":119,"sourceCodeEnd":142,"githubUrl":"https://github.com/apple/pkl/blob/f3efcbfc9b60d30053b0536d664948d7aa1b8673/pkl-cli/src/main/kotlin/org/pkl/cli/CliTestRunner.kt#L119-L142","documentation":"After running all test modules, if any test failed, evalTest throws CliTestException with the \"testsFailed\" message and a non-zero exit code (1, or 10 when an example-failure snapshot was written). This is the CLI's normal signal that the test suite did not pass; it is not a malfunction of the tool itself.","triggerScenarios":"Running `pkl test <modules>` where at least one test assertion fails; the summary reporter reports failures and `failed` is true at the end of evalTest.","commonSituations":"A code change broke expected Pkl values rendered in tests; expected example output is stale after intentional changes; CI run picks up a failing test authored earlier.","solutions":["Read the test summary output to find failing tests and update the code or expected values accordingly.","If output changes are intentional, regenerate/update the expected example files (exit code 10 indicates example-written failures where new output was written for inspection).","Re-run `pkl test` locally before pushing to confirm all tests pass."],"exampleFix":"// before: expected output in test\noutput { text = \"v1\" }\n\n// after intentional change: update the expectation\noutput { text = \"v2\" }","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"// run pkl test and branch on exit codes (1 = failures, 10 = example output written)\nval proc = ProcessBuilder(\"pkl\", \"test\", *modules).start()\nval code = proc.waitFor()\nwhen (code) {\n  0 -> \"\" // all passed\n  10 -> println(\"example outputs were rewritten; review and commit them\")\n  else -> error(\"pkl test failed with exit code $code\")\n}","preventionTips":["Run `pkl test` locally before every commit/push.","After intentional output changes, regenerate and review example files, then rerun tests.","Keep expected outputs in version control so failures are diffable."],"tags":["testing","test-failure","cli","exit-code"],"backgroundTag":"tests-failed","analyzedSha":"f3efcbfc9b60d30053b0536d664948d7aa1b8673","analyzedAt":"2026-09-08T13:10:45.570Z","contentChangedAt":"2026-09-08T13:10:45.570Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}