{"record":{"id":"7e02c96980e0f581","repo":"apache/seatunnel","slug":"table-dataset-path-open-exception","errorCode":"TABLE_DATASET_PATH_OPEN_EXCEPTION","errorMessage":"Failed to create dataset: ","messagePattern":"Failed to create dataset: ","errorType":"error_code","errorClass":"LanceConnectorException","httpStatus":null,"severity":"critical","filePath":"seatunnel-connectors-v2/connector-lance/src/main/java/org/apache/seatunnel/connectors/seatunnel/lance/sink/LanceSinkWriter.java","lineNumber":114,"sourceCode":"        } catch (Exception e) {\n            this.schema = SchemaUtils.convertSchema(firstElement, seaTunnelRowType);\n\n            try {\n                Dataset.create(\n                        allocator,\n                        config.getDatasetPath(),\n                        schema,\n                        new WriteParams.Builder()\n                                .withMaxBytesPerFile(config.getMaxBytesPerFile())\n                                .withMaxRowsPerFile(config.getMaxRowsPerFile())\n                                .withMode(config.getMode())\n                                .withStorageOptions(config.getStorageOptions())\n                                .build());\n\n                this.dataset = Dataset.open(config.getDatasetPath(), allocator);\n                datasetInitialized = true;\n            } catch (Exception createEx) {\n                throw new LanceConnectorException(\n                        LanceConnectorErrorCode.TABLE_DATASET_PATH_OPEN_EXCEPTION,\n                        \"Failed to create dataset: \" + createEx.getMessage(),\n                        createEx);\n            }\n        }\n    }\n\n    @Override\n    public void write(SeaTunnelRow element) throws IOException {\n        if (!datasetInitialized) {\n            initializeDataset(element);\n        }\n\n        batchBuffer.add(element);\n\n        if (batchBuffer.size() >= batchSize) {\n            flushBatch();\n        }","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-lance/src/main/java/org/apache/seatunnel/connectors/seatunnel/lance/sink/LanceSinkWriter.java#L96-L132","documentation":"LanceSinkWriter.initializeDataset tries Dataset.open on the configured dataset path; if that fails it attempts to create the dataset with Dataset.create and reopen it. If the create/reopen path throws (bad URI, storage auth failure, existing incompatible dataset, invalid write params), the error is wrapped as LanceConnectorException with code TABLE_DATASET_PATH_OPEN_EXCEPTION. It means the sink could not initialize the target Lance dataset at datasetPath.","triggerScenarios":"First write() triggers initializeDataset: Dataset.open fails (dataset absent) and the fallback Dataset.create/Dataset.open also fails — e.g. invalid or non-URI datasetPath, missing storage options (S3 credentials/endpoint), object store returning access denied, createMode conflicting with an existing dataset, or corrupt/incompatible existing schema.","commonSituations":"Typo or missing scheme in the 'lance.dataset-path' config (local path vs s3://); storage options omitted in environments where the Lance namespace required them; job restarted against a dataset created with a different schema; bucket/credentials not configured for the worker pods.","solutions":["Check the 'dataset-path' value: it must be a valid Lance URI (absolute local path or s3://...), and the create exception message names the real cause.","Add/verify storage options (endpoint, access key, secret, region) in the sink config so Dataset.create can reach the object store.","Verify write permissions on the target location/bucket.","If the dataset exists but open failed, compare its Arrow schema with the incoming SeaTunnelRowType for incompatibilities; recreate the dataset or align schemas."],"exampleFix":"// before\n\"lance.dataset-path\" = \"my-bucket/tables/users\"  // no scheme, storage options missing\n// after\n\"lance.dataset-path\" = \"s3://my-bucket/tables/users\"\n\"lance.storage.options\" = { \"s3.endpoint\": \"...\", \"s3.access_key_id\": \"...\", \"s3.secret_access_key\": \"...\" }","handlingStrategy":"try-catch","validationCode":"// validate config before job submission\nassert config.getDatasetPath() != null && (config.getDatasetPath().startsWith(\"s3://\") || new java.io.File(config.getDatasetPath()).isAbsolute());\nassert config.getStorageOptions() != null; // when path is remote","typeGuard":null,"tryCatchPattern":"try { writer.write(row); } catch (LanceConnectorException e) { if (e.getErrorCode() == LanceConnectorErrorCode.TABLE_DATASET_PATH_OPEN_EXCEPTION) { log.error(\"dataset init failed at {}: {}\", datasetPath, e.getCause(), e); throw e; } }","preventionTips":["Use fully-qualified URIs (s3://bucket/path or absolute local path) for dataset-path.","Always supply storage options (endpoint, keys, region) for remote paths.","Pre-create the dataset via catalog.createTable so initializeDataset takes the open path.","Verify worker-level access to the object store before launching the job."],"tags":["lance","sink","dataset","storage","config"],"backgroundTag":"file-open-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}