{"record":{"id":"48a66352283184ad","repo":"apache/beam","slug":"duplicate-column-names-found-duplicates","errorCode":null,"errorMessage":"Duplicate column names found: {duplicates}","messagePattern":"Duplicate column names found: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/rag/ingestion/mysql.py","lineNumber":109,"sourceCode":"class _MySQLQueryBuilder:\n  def __init__(\n      self,\n      table_name: str,\n      *,\n      column_specs: list[ColumnSpec],\n      conflict_resolution: Optional[ConflictResolution] = None):\n    \"\"\"Builds SQL queries for writing EmbeddableItems with Embeddings to MySQL.\n    \"\"\"\n    self.table_name = table_name\n\n    self.column_specs = column_specs\n    self.conflict_resolution_strategy = _create_conflict_strategy(\n        conflict_resolution)\n\n    names = [col.column_name for col in self.column_specs]\n    duplicates = set(name for name in names if names.count(name) > 1)\n    if duplicates:\n      raise ValueError(f\"Duplicate column names found: {duplicates}\")\n\n    fields = [(col.column_name, col.python_type) for col in self.column_specs]\n    type_name = f\"VectorRecord_{table_name}\"\n    self.record_type = NamedTuple(type_name, fields)  # type: ignore\n\n    registry.register_coder(self.record_type, RowCoder)\n\n  def build_insert(self) -> str:\n    fields = [col.column_name for col in self.column_specs]\n    placeholders = [col.placeholder for col in self.column_specs]\n\n    # Build base query\n    query = f\"\"\"\n        INSERT INTO {self.table_name}\n        ({', '.join(fields)})\n        VALUES ({', '.join(placeholders)})\n    \"\"\"\n    conflict_clause = self.conflict_resolution_strategy.get_conflict_clause(","sourceCodeStart":91,"sourceCodeEnd":127,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/rag/ingestion/mysql.py#L91-L127","documentation":"MySqlVectorWriterConfig.__init__ builds a NamedTuple record type from the ColumnSpec list, so each column name must be unique. If two specs share a column_name, the generated record type and SQL insert would be ambiguous, so it fails fast.","triggerScenarios":"Passing column_specs where two ColumnSpec entries have the same column_name, e.g. a builder adding both a default id column and an explicit id column in MySqlVectorWriterConfig.","commonSituations":"Appending specs in a loop that reuses a fixed column name; combining metadata and vector fields that both map to 'content' or 'id'; merging two spec lists without de-duplicating.","solutions":["Rename one of the duplicated column_name values in your ColumnSpec list.","Print [c.column_name for c in column_specs] and de-duplicate before constructing the writer config.","If two columns need different values but share a DB name, use distinct spec names or adjust the table schema."],"exampleFix":"// before\nspecs = [ColumnSpec(id), ColumnSpec.vector(\"embedding\"), ColumnSpec(id)]\n// after\nspecs = [ColumnSpec(id), ColumnSpec.vector(\"embedding\")]","handlingStrategy":"validation","validationCode":"names = [c.column_name for c in column_specs]\nassert len(names) == len(set(names)), f\"duplicate columns: {set(n for n in names if names.count(n) > 1)}\"","typeGuard":"def columns_unique(specs) -> bool:\n    names = [c.column_name for c in specs]\n    return len(names) == len(set(names))","tryCatchPattern":"try:\n    cfg = MySqlVectorWriterConfig(..., column_specs=specs)\nexcept ValueError as e:\n    if \"Duplicate column names\" in str(e):\n        seen = set(); specs = [s for s in specs if not (s.column_name in seen or seen.add(s.column_name))]\n        cfg = MySqlVectorWriterConfig(..., column_specs=specs)\n    else:\n        raise","preventionTips":["De-duplicate specs before building the writer config","Use a schema definition list as the single source of truth for column names","Add a startup test that constructs the writer config with your production spec list"],"tags":["python","mysql","rag","schema-validation"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}