{"record":{"id":"ada13e563a2c0a84","repo":"apache/beam","slug":"insert-failed","errorCode":null,"errorMessage":"Insert failed: ","messagePattern":"Insert failed: ","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java","lineNumber":1293,"sourceCode":"                      rows,\n                      maxThrottlingMsec,\n                      sleeper,\n                      streamingInsertsResults)));\n          strideIndices.add(strideIndex);\n          retTotalDataSize += dataSize;\n          rows = new ArrayList<>();\n        }\n\n        try {\n          for (int i = 0; i < futures.size(); i++) {\n            List<TableDataInsertAllResponse.InsertErrors> errors = futures.get(i).get();\n            if (errors == null) {\n              continue;\n            }\n\n            for (TableDataInsertAllResponse.InsertErrors error : errors) {\n              if (error.getIndex() == null) {\n                throw new IOException(\"Insert failed: \" + error + \", other errors: \" + allErrors);\n              }\n              int errorIndex = error.getIndex().intValue() + strideIndices.get(i);\n              failedIndices.add(errorIndex);\n              if (retryPolicy.shouldRetry(new InsertRetryPolicy.Context(error))) {\n                allErrors.add(error);\n                retryRows.add(rowsToPublish.get(errorIndex));\n                // TODO (https://github.com/apache/beam/issues/20891): Select the retry rows(using\n                // errorIndex) from the batch of rows which attempted insertion in this call.\n                // Not the entire set of rows in rowsToPublish.\n                if (retryIds != null) {\n                  // retryIds is non-null exactly when idsToPublish is non-null; see where both are\n                  // initialized above.\n                  retryIds.add(checkStateNotNull(idsToPublish).get(errorIndex));\n                }\n              } else {\n                numFailedRows += 1;\n                errorContainer.add(failedInserts, error, ref, rowsToPublish.get(errorIndex));\n              }","sourceCodeStart":1275,"sourceCodeEnd":1311,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java#L1275-L1311","documentation":"Thrown during streaming insertAll result processing when BigQuery returns per-row InsertErrors and one error row has a null index. A null index means the error applies to the whole request rather than a specific row (e.g. table-level failure), so the code cannot attribute it; it throws an IOException listing the error and all other collected errors.","triggerScenarios":"BigQuery returns TableDataInsertAllResponse.InsertErrors with getIndex()==null — typically when the entire insert request failed: table does not exist (or was deleted mid-run), schema mismatch, or access/quota error at the request level.","commonSituations":"Destination table deleted or recreated while a streaming write is in flight; streaming buffer/schema conflicts after a schema change; wrong table reference in dynamic destinations.","solutions":["Verify the destination table exists and its schema matches the rows being written","Check the allErrors content in the message for the underlying request-level cause (notFound, invalid schema, access denied)","Recreate or re-point the table and re-run; avoid concurrent schema mutations during writes","Wrap the write with BigQueryIO's retry policy / DLQ for request-level failures"],"exampleFix":"// before\nbq.tables().delete(project, dataset, table).execute(); // while pipeline streams into it\n// after\n// stop the pipeline or write to a new table before deleting the old one","handlingStrategy":"retry","validationCode":"// before streaming, verify table exists and schema matches\nTable table = datasetService.getTable(ref);\nif (table == null || !schemaMatches(table.getDefinition().getSchema(), rowSchema)) {\n  throw new IllegalStateException(\"Destination table missing or schema mismatch\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  writeResult = rows.apply(BigQueryIO.writeTableRows()...);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Insert failed:\")) {\n    // parse request-level error; recreate/verify table, then re-run\n  }\n}","preventionTips":["Don't delete or recreate destination tables while writes are in flight","Validate row schema against the table schema before writing","Use deterministic row IDs for idempotent re-runs","Watch for concurrent schema migrations"],"tags":["bigquery","gcp","streaming-insert","table-not-found"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}