tensorflow/models · error · ValueError

data_folder must be set as the downloaded folder path.

Error message

data_folder must be set as the downloaded folder path.

What it means

Error "data_folder must be set as the downloaded folder path." thrown in tensorflow/models.

Source

Thrown at official/projects/nhnet/raw_data_process.py:70

  story_filepath = os.path.join(FLAGS.data_folder, filename + ".json")
  output_folder = os.path.join(FLAGS.data_folder, "processed")
  os.makedirs(output_folder, exist_ok=True)
  output_filepaths = []
  for i in range(FLAGS.num_tfrecords_shards):
    output_filepaths.append(
        os.path.join(
            output_folder, "%s.tfrecord-%.5d-of-%.5d" %
            (filename, i, FLAGS.num_tfrecords_shards)))
  (total_num_examples,
   generated_num_examples) = data_processor.generate_examples(
       story_filepath, output_filepaths)
  print("For %s, %d examples have been generated from %d stories in json." %
        (filename, generated_num_examples, total_num_examples))


def main(_):
  if not FLAGS.data_folder:
    raise ValueError("data_folder must be set as the downloaded folder path.")
  if not FLAGS.vocab:
    raise ValueError("vocab must be set as the filepath of BERT vocabulary.")
  data_processor = raw_data_processor.RawDataProcessor(
      vocab=FLAGS.vocab,
      do_lower_case=FLAGS.do_lower_case,
      len_title=FLAGS.len_title,
      len_passage=FLAGS.len_passage,
      max_num_articles=FLAGS.max_num_articles,
      include_article_title_in_passage=FLAGS.include_article_title_in_passage,
      include_text_snippet_in_example=True)
  print("Loading crawled articles...")
  num_articles = data_processor.read_crawled_articles(FLAGS.crawled_articles)
  print("Total number of articles loaded: %d" % num_articles)
  print()
  transform_as_tfrecords(data_processor, "train")
  transform_as_tfrecords(data_processor, "valid")
  transform_as_tfrecords(data_processor, "test")

View on GitHub (pinned to e006f5f0d5)

When it happens

Trigger: Thrown at official/projects/nhnet/raw_data_process.py:70 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/0d7dcefb90ec209e. Report an issue: GitHub.