{"record":{"id":"c4d2e71b4c24fef9","repo":"xai-org/x-algorithm","slug":"did-not-find-any-files-matching-file-pattern","errorCode":null,"errorMessage":"Did not find any files matching {file_pattern}","messagePattern":"Did not find any files matching (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"adult-content/dataset_utils.py","lineNumber":51,"sourceCode":"\n    if features_as_dict:\n        features = {\"media_embedding\": embedding}\n    else:\n        features = embedding\n\n    return features, label\n\n\ndef build_dataset(\n    dataset_path, embedding_dim, batch_size, do_resample=False, do_repeat=False\n):\n    file_pattern = f\"{dataset_path}/*.tfrecord\"\n    files = tf.io.gfile.glob(file_pattern)\n\n    random.shuffle(files)\n\n    if not len(files):\n        raise ValueError(f\"Did not find any files matching {file_pattern}\")\n\n    ds = tf.data.TFRecordDataset(files).map(\n        lambda x: decode_fn_embedding(x, embedding_dim)\n    )\n    ds = ds.map(lambda x: preprocess_embedding_example(x, positive_label=1))\n\n    if do_resample:\n        ds = ds.apply(resample_fn).map(lambda _, b: (b))\n\n    ds = ds.batch(batch_size=batch_size)\n\n    if do_repeat:\n        ds = ds.shuffle(buffer_size=10).repeat()\n\n    return ds\n","sourceCodeStart":33,"sourceCodeEnd":67,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/adult-content/dataset_utils.py#L33-L67","documentation":"build_dataset globs '{dataset_path}/*.tfrecord' with tf.io.gfile.glob and raises ValueError when the result is empty, i.e. the directory holds no TFRecord files (or the path/glob resolves nowhere on the local or remote filesystem).","triggerScenarios":"Calling build_dataset(dataset_path=...) on a directory with no .tfrecord files; wrong dataset_path (typo, missing shard prefix); files present but named differently (.tfrecords, .record); GCS/S3 path with wrong bucket/prefix so glob returns an empty list.","commonSituations":"Pointing train_model at an extraction step that failed or wrote elsewhere; extension convention mismatch; empty shards directory after a partial upload; calling build_dataset before dataset generation completed.","solutions":["List the directory (tf.io.gfile.listdir) and confirm .tfrecord files exist at that exact path","Fix dataset_path to the directory that actually contains the shards","If files use a different extension, rename them or adjust the pattern","Re-run the dataset extraction/generation step that should have produced the tfrecords"],"exampleFix":"# before\nbuild_dataset('/gs/adult-content/train', ...)\n# after\nbuild_dataset('/gs/adult-content/train_shards', ...)  # dir actually containing *.tfrecord","handlingStrategy":"validation","validationCode":"files = tf.io.gfile.glob(f\"{dataset_path}/*.tfrecord\")\nassert files, f\"no tfrecords under {dataset_path}\"","typeGuard":"def has_tfrecords(path: str) -> bool:\n    return len(tf.io.gfile.glob(f\"{path}/*.tfrecord\")) > 0","tryCatchPattern":"try:\n    ds = build_dataset(path, ...)\nexcept ValueError as e:\n    if 'Did not find any files' in str(e):\n        path = locate_dataset(); ds = build_dataset(path, ...)\n    else:\n        raise","preventionTips":["Assert dataset dirs are non-empty in data-validation CI","Name shards with the canonical .tfrecord extension","Have generation jobs emit a _SUCCESS marker checked before training"],"tags":["python","tensorflow","dataset","file-not-found"],"backgroundTag":"empty-file-glob","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}