{"record":{"id":"a295942cccf09414","repo":"apache/beam","slug":"could-not-create-destination-for-extract-job-s","errorCode":null,"errorMessage":"Could not create destination for extract job %s","messagePattern":"Could not create destination for extract job (.+?)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/FakeJobService.java","lineNumber":559,"sourceCode":"\n  private void writeRowsHelper(\n      List<TableRow> rows, Schema avroSchema, String destinationPattern, int shard) {\n    String filename = destinationPattern.replace(\"*\", String.format(\"%012d\", shard));\n    try (WritableByteChannel channel =\n            FileSystems.create(\n                FileSystems.matchNewResource(filename, false /* isDirectory */), MimeTypes.BINARY);\n        DataFileWriter<GenericRecord> tableRowWriter =\n            new DataFileWriter<>(new GenericDatumWriter<GenericRecord>(avroSchema))\n                .create(avroSchema, Channels.newOutputStream(channel))) {\n      for (Map<String, Object> record : rows) {\n        GenericRecordBuilder genericRecordBuilder = new GenericRecordBuilder(avroSchema);\n        for (Map.Entry<String, Object> field : record.entrySet()) {\n          genericRecordBuilder.set(field.getKey(), field.getValue());\n        }\n        tableRowWriter.append(genericRecordBuilder.build());\n      }\n    } catch (IOException e) {\n      throw new IllegalStateException(\n          String.format(\"Could not create destination for extract job %s\", filename), e);\n    }\n  }\n}\n","sourceCodeStart":541,"sourceCodeEnd":564,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/FakeJobService.java#L541-L564","documentation":"Thrown by FakeJobService's writeRowsHelper, a Beam testing fake that emulates BigQuery. When writing extracted records to a temp file, an IOException while creating/appending the destination file is wrapped in this IllegalStateException. It indicates the fake extract job could not materialize its output file.","triggerScenarios":"Calling writeRows/writeRowsHelper in FakeJobService when the target file (named by `filename`) cannot be created or opened for writing — e.g. bad path, missing parent directory, or disk full.","commonSituations":"Running Beam BigQuery I/O integration tests on machines with restricted temp directories, read-only /tmp, or tests that clean up temp folders concurrently while the fake job service is still writing.","solutions":["Check that the parent directory of `filename` exists and is writable before running the test","Verify the test environment's temp directory (java.io.tmpdir) is writable and has free space","Look at the cause IOException in the stack trace for the underlying filesystem error","Ensure no concurrent test cleanup deletes the temp file while the extract job writes it"],"exampleFix":"// before\nFile out = new File(unverifiedPath);\n// after\nFile out = new File(path);\nFiles.createDirectories(out.getParentFile().toPath());\nif (!out.canWrite()) { throw new SkipException(\"temp dir not writable\"); }","handlingStrategy":"try-catch","validationCode":"File dest = new File(filename);\nif (dest.getParentFile() != null && !dest.getParentFile().exists()) {\n  Files.createDirectories(dest.getParentFile().toPath());\n}\nif (!Files.isWritable(dest.getParentFile() != null ? dest.getParentFile().toPath() : Paths.get(\".\"))) {\n  throw new IllegalStateException(\"destination dir not writable: \" + filename);\n}","typeGuard":null,"tryCatchPattern":"try {\n  writeRows(rows, filename);\n} catch (IllegalStateException e) {\n  LOGGER.severe(\"extract destination failed: \" + filename + \" cause=\" + e.getCause());\n  throw e;\n}","preventionTips":["Pre-create temp directories in test setup with Files.createDirectories","Set java.io.tmpdir to a writable location in CI","Avoid deleting temp files concurrently with the fake job service"],"tags":["java","io","bigquery","testing","file-write"],"backgroundTag":"file-write-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}