apache/beam · error

scheme already registered

Error message

scheme %v already registered

What it means

filesystem.Register panics when the same URL scheme (e.g. "gs", "s3", "hdfs") is registered more than once. The package keeps a global scheme->factory registry and treats a duplicate registration as an initialization bug that could silently swap filesystem implementations.

Solutions

1. Remove the duplicate filesystem.Register call, keeping a single registration per scheme.
2. Move registration into an init() of exactly one package so it runs once per process.
3. If dynamic registration is needed, guard with your own sync.Once or check the registry first.
4. In tests, register the scheme once in TestMain rather than in each benchmark/test function.

Example fix

// before
func setup() { filesystem.Register("s3", newS3FS) } // called per test

// after
var once sync.Once
func setup() {
    once.Do(func() { filesystem.Register("s3", newS3FS) })
}

if _, already := filesystem.DefaultRegistry(); already {
    // skip re-registration
}

var registerOnce sync.Once
func registerFS(scheme string, fs func(context.Context) filesystem.Interface) {
    registerOnce.Do(func() {
        defer func() {
            if r := recover(); r != nil {
                log.Printf("filesystem register skipped: %v", r)
            }
        }()
        filesystem.Register(scheme, fs)
    })
}

When it happens

Trigger: Calling filesystem.Register("s3", myFs) twice in the same process — e.g. in two init() functions across packages, or in a library that users import alongside a package that registers the same scheme.

Common situations: Two vendored/internal packages both registering the same scheme; copy-pasted init() registration in a test helper and production package; dynamically calling Register inside a function that can run multiple times (tests, benchmarks, repeated setup).

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• errorForMissingScheme(scheme, path)
• type already registered for
• AfterProcessingTime trigger set without a delay or…
• At least one subtrigger required for composite triggers.
• attempted to add namespace to missing coder id

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6333defb10788544. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/filesystem/filesystem.go:52

)

var registry = make(map[string]func(context.Context) Interface)

// wellKnownSchemeImportPaths is used for delivering useful error messages when a
// scheme is not found.
var wellKnownSchemeImportPaths = map[string]string{
	"memfs":   "github.com/apache/beam/sdks/v2/go/pkg/beam/io/filesystem/memfs",
	"default": "github.com/apache/beam/sdks/v2/go/pkg/beam/io/filesystem/local",
	"gs":      "github.com/apache/beam/sdks/v2/go/pkg/beam/io/filesystem/gcs",
	"s3":      "github.com/apache/beam/sdks/v2/go/pkg/beam/io/filesystem/s3",
}

// Register registers a file system backend under the given scheme.  For
// example, "hdfs" would be registered a HFDS file system and HDFS paths used
// transparently.
func Register(scheme string, fs func(context.Context) Interface) {
	if _, ok := registry[scheme]; ok {
		panic(fmt.Sprintf("scheme %v already registered", scheme))
	}
	registry[scheme] = fs
}

// New returns a new Interface for the given file path's scheme.
func New(ctx context.Context, path string) (Interface, error) {
	scheme := getScheme(path)
	mkfs, ok := registry[scheme]
	if !ok {
		return nil, errorForMissingScheme(scheme, path)
	}
	return mkfs(ctx), nil
}

func errorForMissingScheme(scheme, path string) error {
	messageSuffix := ""
	if suggestedImportPath, ok := wellKnownSchemeImportPaths[scheme]; ok {
		messageSuffix = fmt.Sprintf(": Consider adding the following import to your program to register an implementation for %q:\n  import _ %q", scheme, suggestedImportPath)

View on GitHub (pinned to 12126d8942)