{"record":{"id":"b090fe8318068a53","repo":"apple/pkl","slug":"cannotevaluatenonfilebasedtestmodule","errorCode":"cannotEvaluateNonFileBasedTestModule","errorMessage":"cannotEvaluateNonFileBasedTestModule","messagePattern":"cannotEvaluateNonFileBasedTestModule","errorType":"error_code","errorClass":"VmException","httpStatus":null,"severity":"error","filePath":"pkl-core/src/main/java/org/pkl/core/runtime/TestRunner.java","lineNumber":145,"sourceCode":"                          err.getMessage(), err.toPklException(stackFrameTransformer, useColor));\n                  resultBuilder.addError(error);\n                }\n                return true;\n              });\n          testResults.add(resultBuilder.build());\n          return true;\n        });\n    return new TestSectionResults(TestSectionName.FACTS, Collections.unmodifiableList(testResults));\n  }\n\n  private TestSectionResults runExamples(VmTyped testModule, ModuleInfo info) {\n    var examples = VmUtils.readMember(testModule, Identifier.EXAMPLES);\n    if (examples instanceof VmNull)\n      return new TestSectionResults(TestSectionName.EXAMPLES, List.of());\n\n    var moduleUri = info.getModuleKey().getUri();\n    if (!moduleUri.getScheme().equalsIgnoreCase(\"file\")) {\n      throw new VmExceptionBuilder()\n          .evalError(\"cannotEvaluateNonFileBasedTestModule\", moduleUri)\n          .build();\n    }\n\n    var examplesMapping = (VmMapping) examples;\n    var moduleFile = Path.of(moduleUri);\n    var expectedOutputFile = moduleFile.resolveSibling(moduleFile.getFileName() + \"-expected.pcf\");\n    var actualOutputFile = moduleFile.resolveSibling(moduleFile.getFileName() + \"-actual.pcf\");\n\n    try {\n      Files.deleteIfExists(actualOutputFile);\n    } catch (IOException e) {\n      throw new VmExceptionBuilder()\n          .evalError(\"ioErrorWritingTestOutputFile\", actualOutputFile)\n          .withCause(e)\n          .build();\n    }\n    try {","sourceCodeStart":127,"sourceCodeEnd":163,"githubUrl":"https://github.com/apple/pkl/blob/f3efcbfc9b60d30053b0536d664948d7aa1b8673/pkl-core/src/main/java/org/pkl/core/runtime/TestRunner.java#L127-L163","documentation":"Pkl's `examples` test block can only be evaluated for modules loaded from the local filesystem, because the test runner must read and write sibling `*-expected.pcf` / `*-actual.pcf` files next to the module file. When `TestRunner.runExamples` finds the module URI's scheme is not `file` (e.g. a module from a JAR, HTTP, or an in-memory/synthetic module), it throws this error instead of attempting file I/O on a non-file path.","triggerScenarios":"Running `pkl test` (or TestRunner.run) against a module whose ModuleKey URI scheme is not `file` — e.g. a module resolved from an HTTP(S) URL, a classpath/JAR resource, or a synthetic in-memory module.","commonSituations":"Testing modules fetched from a remote registry or URL; testing modules embedded in an application via custom module resolvers; CI setups that load Pkl code over the network; running tests on synthetic modules built programmatically with the Embedding API.","solutions":["Materialize the module to a real file on disk and run the test against that file path.","If the module comes from a remote source, download it into the project directory and reference it by relative path.","If using a custom ModuleResolver/synthetic ModuleKey, switch the test module to a FileModuleKey pointing at a temporary file.","Extract the `examples` block and validate it manually with `pkl eval` if filesystem-backed testing is not possible."],"exampleFix":"// before: pkl test https://example.com/schema/MySchema.pkl\n// after\ncurl -o MySchema.pkl https://example.com/schema/MySchema.pkl\npkl test MySchema.pkl","handlingStrategy":"validation","validationCode":"// check module scheme before running tests\nURI uri = moduleKey.getUri();\nif (!\"file\".equalsIgnoreCase(uri.getScheme())) {\n    throw new IllegalArgumentException(\"pkl test requires a file-based module, got: \" + uri);\n}","typeGuard":"boolean isFileBasedModule(ModuleKey key) { return \"file\".equalsIgnoreCase(key.getUri().getScheme()); }","tryCatchPattern":null,"preventionTips":["Always materialize remote/generated modules to disk before running `pkl test`.","Document that test modules must live in the project filesystem, not behind custom resolvers.","Validate module URIs in CI before invoking the test runner."],"tags":["pkl","testing","filesystem","module-loading"],"backgroundTag":"invalid-url","analyzedSha":"f3efcbfc9b60d30053b0536d664948d7aa1b8673","analyzedAt":"2026-09-08T13:10:45.570Z","contentChangedAt":"2026-09-08T13:10:45.570Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}