{"record":{"id":"7ec34ea71b6c4725","repo":"apache/beam","slug":"a-schema-must-be-provided-when-writing-to-bigquery-using","errorCode":null,"errorMessage":"A schema must be provided when writing to BigQuery using Avro based file loads","messagePattern":"A schema must be provided when writing to BigQuery using Avro based file loads","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery.py","lineNumber":2414,"sourceCode":"          max_insert_payload_size=self._max_insert_payload_size,\n          max_retries=self._max_retries,\n          num_streaming_keys=self._num_streaming_keys)\n\n      return WriteResult(\n          method=WriteToBigQuery.Method.STREAMING_INSERTS,\n          failed_rows=outputs[BigQueryWriteFn.FAILED_ROWS],\n          failed_rows_with_errors=outputs[\n              BigQueryWriteFn.FAILED_ROWS_WITH_ERRORS])\n\n    elif method_to_use == WriteToBigQuery.Method.FILE_LOADS:\n      if self._temp_file_format == bigquery_tools.FileFormat.AVRO:\n        if self.schema == SCHEMA_AUTODETECT:\n          raise ValueError(\n              'Schema auto-detection is not supported when using Avro based '\n              'file loads into BigQuery. Please specify a schema or set '\n              'temp_file_format=\"NEWLINE_DELIMITED_JSON\"')\n        if self.schema is None:\n          raise ValueError(\n              'A schema must be provided when writing to BigQuery using '\n              'Avro based file loads')\n\n      if self.schema and type(self.schema) is dict:\n\n        def find_in_nested_dict(schema):\n          for field in schema['fields']:\n            if field['type'] == 'JSON':\n              logging.warning(\n                  'Found JSON type in TableSchema for \"File_LOADS\" write '\n                  'method. Make sure the TableSchema field is a parsed '\n                  'JSON to ensure the read as a JSON type. Otherwise it '\n                  'will read as a raw (escaped) string.')\n            elif field['type'] == 'STRUCT':\n              find_in_nested_dict(field)\n\n        find_in_nested_dict(self.schema)\n","sourceCodeStart":2396,"sourceCodeEnd":2432,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery.py#L2396-L2432","documentation":"With FILE_LOADS and Avro temp file format, expand() requires a schema to be present; schema=None means Beam cannot build the Avro load job. This is the companion check to auto-detection: with Avro, some explicit schema must be supplied.","triggerScenarios":"WriteToBigQuery(method=FILE_LOADS, schema=None, temp_file_format=Avro) expanded on input whose element type cannot yield a schema.","commonSituations":"Forgetting to pass schema when the input is plain JSON dicts; refactoring removed the schema parameter; defaults changed between Beam versions.","solutions":["Pass an explicit schema argument (BigQuery schema string, dict, or callable).","Set temp_file_format='NEWLINE_DELIMITED_JSON' to allow schema auto-detection from untyped data.","Feed a typed PCollection (Beam schema rows) so a schema can be derived."],"exampleFix":"// before\nWriteToBigQuery(table='proj:ds.tbl', method='FILE_LOADS', temp_file_format='AVRO')\n// after\nWriteToBigQuery(table='proj:ds.tbl', method='FILE_LOADS', temp_file_format='AVRO', schema={'fields': [{'name': 'id', 'type': 'STRING'}]})","handlingStrategy":"validation","validationCode":"if temp_file_format == 'AVRO' and schema is None:\n    raise ValueError('schema is required for Avro file loads')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Make schema a mandatory part of your pipeline config for FILE_LOADS.","Fall back to NEWLINE_DELIMITED_JSON when no schema is known."],"tags":["apache-beam","bigquery","avro","missing-schema"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}