{"record":{"id":"b46f4683eb18ba41","repo":"apache/beam","slug":"unable-to-find-value-index","errorCode":null,"errorMessage":"Unable to find value #${index}","messagePattern":"Unable to find value #(.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java","lineNumber":536,"sourceCode":"      this.inputSchema = inputSchema;\n      this.referencedColumns = new TreeSet<>();\n    }\n\n    FieldAccessDescriptor getFieldAccess() {\n      return FieldAccessDescriptor.withFieldIds(this.referencedColumns);\n    }\n\n    @Override\n    public Expression field(BlockBuilder list, int index, Type storageType) {\n      this.referencedColumns.add(index);\n      return getBeamField(list, index, input, inputSchema, true);\n    }\n\n    // Read field from Beam Row\n    private static Expression getBeamField(\n        BlockBuilder list, int index, Expression input, Schema schema, boolean useByteString) {\n      if (index >= schema.getFieldCount() || index < 0) {\n        throw new IllegalArgumentException(\"Unable to find value #\" + index);\n      }\n\n      final Expression expression = list.append(list.newName(\"current\"), input);\n      final Field field = schema.getField(index);\n      final FieldType fieldType = field.getType();\n      final Expression fieldName = Expressions.constant(field.getName());\n      Expression value = getBeamField(list, expression, fieldName, fieldType);\n\n      return toCalciteValue(value, fieldType, useByteString);\n    }\n\n    private static Expression getBeamField(\n        BlockBuilder list, Expression expression, Expression fieldName, FieldType fieldType) {\n      final Expression value;\n      switch (fieldType.getTypeName()) {\n        case BYTE:\n          return Expressions.call(expression, \"getByte\", fieldName);\n        case INT16:","sourceCodeStart":518,"sourceCodeEnd":554,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java#L518-L554","documentation":"While generating code to read a field from a Beam Row inside the Calc operator, getBeamField validates the column index against the schema's field count. An index outside [0, fieldCount) means the generated code and the row schema disagree, so IllegalArgumentException is thrown. This is an internal consistency check between Calcite's row type and the Beam input schema.","triggerScenarios":"Running a SQL query where the Calcite plan references a column ordinal that the incoming Beam Row schema doesn't have — typically after a mismatched input PCollection schema, aliased/renamed columns in a subquery, or a codegen bug.","commonSituations":"Passing a PCollection with fewer fields than the query expects (wrong table registered to the same name); schema evolved upstream (column dropped) while the query still references it; UNION/subquery column-count mismatches.","solutions":["Verify the PCollection registered for the table has a schema matching what the query selects (names/order/count)","Check for schema drift between pipeline stages (e.g. upstream transform dropped a field)","Simplify the query to isolate which column index is out of range (e.g. `SELECT *` first)","Upgrade Beam if it looks like a codegen bug with a specific query shape"],"exampleFix":"// before\np.apply(\"t\", rowsWithDroppedField).apply(SqlTransform.query(\"SELECT a, b, c FROM t\"));\n// after\n// ensure rowsWithDroppedField schema contains fields a, b, c, or select only existing fields\np.apply(SqlTransform.query(\"SELECT a, b FROM t\"));","handlingStrategy":"validation","validationCode":"boolean schemaMatchesQuery(Schema s, int maxReferencedIndex) {\n  return s.getFieldCount() > maxReferencedIndex;\n}","typeGuard":"boolean columnIndexInBounds(Schema s, int i) {\n  return i >= 0 && i < s.getFieldCount();\n}","tryCatchPattern":"try {\n  rows.apply(SqlTransform.query(sql));\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().startsWith(\"Unable to find value #\")) {\n    // schema/query mismatch: re-check registered table schema\n  }\n}","preventionTips":["Assert input PCollection schema matches query expectations in tests","Use stable table/field names and avoid manual ordinal assumptions","Version-check schemas when upstream stages evolve","Start with SELECT * to validate schema alignment before complex projections"],"tags":["java","apache-beam","sql","schema","index-out-of-bounds"],"backgroundTag":"index-out-of-bounds","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}