apache/beam · error

Unknown type of encoding context

Error message

Unknown type of encoding context

What it means

SchemaTransformDiscovery.discover_config queries an expansion service for available SchemaTransforms and filters them by substring match against the requested name. If no identifier contains the given name, ValueError is raised. This means the requested transform does not exist in the service's catalog.

Source

Thrown at sdks/typescript/src/apache_beam/coders/required_coders.ts:97

   * console.log(w1.finish())  // ==> prints Uint8Array(6) [ 5, 98, 121, 116, 101, 115 ], where 5 is the length prefix.
   * const w2 = new Writer()
   * new BytesCoder().encode("bytes", w1, Context.wholeStream)
   * console.log(w2.finish())  // ==> prints Uint8Array(5) [ 98, 121, 116, 101, 115 ], without the length prefix
   * ```
   * @param value - a byte array to encode. This represents an element to be encoded.
   * @param writer - a writer to access the stream of bytes with encoded data
   * @param context - whether to encode the data with delimiters (`Context.needsDelimiters`), or without (`Context.wholeStream`).
   */
  encode(value: Uint8Array, writer: Writer, context: Context) {
    switch (context) {
      case Context.wholeStream:
        writeRawBytes(value, writer);
        break;
      case Context.needsDelimiters:
        writer.bytes(value);
        break;
      default:
        throw new Error("Unknown type of encoding context");
    }
  }

  /**
   * Decode the input byte stream into a byte array.
   * If context is `needsDelimiters`, the first bytes will be interpreted as a var-int32 encoding
   * the length of the data.
   *
   * If the context is `wholeStream`, the whole input stream is decoded as-is.
   *
   * @param reader - a reader to access the input byte stream
   * @param context - whether the data is encoded with delimiters (`Context.needsDelimiters`), or without (`Context.wholeStream`).
   * @returns
   */
  decode(reader: Reader, context: Context): Uint8Array {
    switch (context) {
      case Context.wholeStream:
        return reader.buf.slice(reader.pos);

View on GitHub (pinned to 12126d8942)

Solutions

  1. Correct the transform name to match (case-sensitively as substring) the SchemaTransform identifier.
  2. List available transforms to see valid names, e.g. iterate external.SchemaTransforms / the catalog returned by the expansion service.
  3. Point expansion_service at a jar that actually contains the desired transform.

Example fix

// before
SchemaTransforms.discover_config('KafKa')  # no match
// after
SchemaTransforms.discover_config('Kafka')  # e.g. 'beam:schematransform:org.apache.beam:kafka_read:v1'
Defensive patterns

Strategy: validation

Validate before calling

catalog = [st.identifier for st in schematransforms]
assert any(name in ident for ident in catalog), f'{name!r} not in {catalog}'

Type guard

def resolves_to_one(name: str, identifiers: list[str]) -> bool:
    return sum(name in i for i in identifiers) == 1

Try / catch

try:
    cfg = SchemaTransforms.discover_config(name, expansion_service=svc)
except ValueError as e:
    logger.error('No SchemaTransform matched %r: %s', name, e)
    cfg = None

Prevention

When it happens

Trigger: Calling discover_config(name) (or XWithSchemaTransforms helpers) where no returned SchemaTransform identifier contains `name` as a substring, e.g. discover_config('BigQueery') or a transform not provided by that expansion service jar.

Common situations: Typos in transform names; expecting Java transforms from a service jar that does not include them; using a stale or too-old expansion service that lacks the transform.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f6bfa81d285a20ec. Report an issue: GitHub.