What causes "Kafka transforms moved to apache_beam.io.kafka" in apache/beam?

Change imports to `from apache_beam.io.kafka import ReadFromKafka` etc.

Kafka transforms moved to apache_beam.io.kafka

Error message

Kafka transforms moved to apache_beam.io.kafka

What it means

Importing apache_beam.io.external.kafka emits a DeprecationWarning because the Kafka transforms (ReadFromKafka, ReadFromKafkaSchema, WriteToKafka, WriteToKafkaSchema) were relocated to apache_beam.io.kafka. The old module is a backward-compatibility re-export shim that warns at import time.

Solutions

Change imports to `from apache_beam.io.kafka import ReadFromKafka` etc.
Run `grep -r 'io.external.kafka' .` and update every occurrence.
Pin/upgrade apache-beam to a version where the new module exists and re-run tests.

Example fix

// before
from apache_beam.io.external.kafka import ReadFromKafka
// after
from apache_beam.io.kafka import ReadFromKafka

Defensive patterns

Strategy: validation

Validate before calling

import warnings
with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter('always')
    import apache_beam.io.external.kafka  # noqa
    if any('moved to apache_beam.io.kafka' in str(x.message) for x in w):
        raise ImportError('Use apache_beam.io.kafka instead')

Prevention

Import Beam IO transforms from apache_beam.io.* top-level modules, not io.external.* shims.
Treat DeprecationWarnings as errors in CI: -W error::DeprecationWarning.
Periodically grep the repo for 'io.external.' imports.

When it happens

Trigger: Any execution of `from apache_beam.io.external.kafka import ReadFromKafka` (or the other three re-exported names) at module import time.

Common situations: Pipelines written against older Beam versions or copied from pre-migration examples/tutorials; dependency upgrades where code still imports from the legacy io.external path.

Understand the failure class

Background: "is deprecated and will be removed" — deprecation warnings for old API names, keywords, and options, and how to migrate before the removal release — this error's family across 29 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fb1a91aca6295be4. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/external/kafka.py:30

# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# For backwards compatibility.

# pytype: skip-file

import warnings

# pylint: disable=unused-import
from apache_beam.io.kafka import ReadFromKafka
from apache_beam.io.kafka import ReadFromKafkaSchema
from apache_beam.io.kafka import WriteToKafka
from apache_beam.io.kafka import WriteToKafkaSchema

warnings.warn(
    "Kafka transforms moved to apache_beam.io.kafka", DeprecationWarning)

View on GitHub (pinned to 12126d8942)