alibaba/DataX · error · ColumnNotFoundException

Col {} not found

Error message

Col {} not found

What it means

ColumnNotFoundException (a Phoenix SQLException subtype) from ThinClientPTable.getColumnForColumnName: the requested column name is not in colMap, the map of columns previously loaded from the server's table metadata. It means the writer is looking up a column that the Phoenix table does not advertise under that exact name.

Source

Thrown at hbase11xsqlwriter/src/main/java/com/alibaba/datax/plugin/writer/hbase11xsqlwriter/ThinClientPTable.java:106

  @Override
  public List<PColumnFamily> getColumnFamilies() {
    throw new UnsupportedOperationException("Not implement");
  }

  @Override
  public PColumnFamily getColumnFamily(byte[] bytes) throws ColumnFamilyNotFoundException {
    throw new UnsupportedOperationException("Not implement");
  }

  @Override
  public PColumnFamily getColumnFamily(String s) throws ColumnFamilyNotFoundException {
    throw new UnsupportedOperationException("Not implement");
  }

  @Override
  public PColumn getColumnForColumnName(String colname) throws ColumnNotFoundException, AmbiguousColumnException {
    if (!colMap.containsKey(colname)) {
      throw new ColumnNotFoundException("Col " + colname + " not found");
    }
    return colMap.get(colname);
  }

  @Override
  public PColumn getColumnForColumnQualifier(byte[] bytes, byte[] bytes1)
      throws ColumnNotFoundException, AmbiguousColumnException {
    throw new UnsupportedOperationException("Not implement");
  }

  @Override
  public PColumn getPKColumn(String s) throws ColumnNotFoundException {
    throw new UnsupportedOperationException("Not implement");
  }

  @Override
  public PRow newRow(KeyValueBuilder keyValueBuilder, long l, ImmutableBytesWritable immutableBytesWritable, boolean b,
      byte[]... bytes) {

View on GitHub (pinned to 80ec23d5c5)

Solutions

1. Compare the job's column list with the server schema (`!describe <table>` in sqlline) and match the exact case.
2. Quote identifiers in DDL/job config consistently so Phoenix's default identifier uppercasing does not bite.
3. Re-load/restart the writer after schema changes so stale metadata caches are refreshed.

Example fix

-- before
CREATE TABLE "events" (id BIGINT PRIMARY KEY, payload VARCHAR)
-- job uses: payload  -> works; job uses: PAYLOAD -> Col not found
-- after
quote consistently or use uppercase identifiers:
CREATE TABLE EVENTS (ID BIGINT PRIMARY KEY, PAYLOAD VARCHAR)

Set<String> schemaCols = new HashSet<>();
for (PColumn c : table.getColumns()) schemaCols.add(c.getName().getString());
for (String col : jobConfigColumns) {
    if (!schemaCols.contains(col)) throw new IllegalArgumentException("Unknown column: " + col);
}

boolean columnExists(ThinClientPTable t, String name) {
    try { t.getColumnForColumnName(name); return true; }
    catch (ColumnNotFoundException e) { return false; }
}

try {
    PColumn c = table.getColumnForColumnName(name);
} catch (ColumnNotFoundException e) {
    // schema drift: refresh table metadata, fix case, or fail with column list diff
}

When it happens

Trigger: Any code path that resolves columns by name against ThinClientPTable (e.g. value conversion in hbase11xsqlwriter) with a column name not present in the loaded schema — unquoted identifiers are uppercased by Phoenix, so lowercase names often miss; schema drift after columns were dropped/renamed also triggers it.

Common situations: Job config uses lowercase column names against a case-sensitive table created with quoted lowercase identifiers (or vice versa); column dropped or renamed server-side; column family-qualified name mismatch; trailing whitespace in column names.

Related errors

• faild load org.apache.phoenix.jdbc.PhoenixDriver
• ColType or colName is null, colType : ${colType} , colName :
• Unknown column type:
• CONFIG_ERROR
• -103

AI-assisted analysis of alibaba/DataX@80ec23d5c5 (2026-08-14). Data as JSON: /api/errors/034b9c57e3de994d. Report an issue: GitHub.