{"record":{"id":"490ee44e005caf5e","repo":"apache/hadoop","slug":"invalid-metadata-files-total-size-fi","errorCode":null,"errorMessage":"Invalid metadata: #files ({}), total_size ({}), filelisturi ({})","messagePattern":"Invalid metadata: #files \\((.+?)\\), total_size \\((.+?)\\), filelisturi \\((.+?)\\)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/GenerateDistCacheData.java","lineNumber":203,"sourceCode":"    // Split the special file that contains the list of distributed cache file\n    // paths and their file sizes such that each split corresponds to\n    // approximately same amount of distributed cache data to be generated.\n    // Consider numTaskTrackers * numMapSlotsPerTracker as the number of maps\n    // for this job, if there is lot of data to be generated.\n    @Override\n    public List<InputSplit> getSplits(JobContext jobCtxt) throws IOException {\n      final JobConf jobConf = new JobConf(jobCtxt.getConfiguration());\n      final JobClient client = new JobClient(jobConf);\n      ClusterStatus stat = client.getClusterStatus(true);\n      int numTrackers = stat.getTaskTrackers();\n      final int fileCount = jobConf.getInt(GRIDMIX_DISTCACHE_FILE_COUNT, -1);\n\n      // Total size of distributed cache files to be generated\n      final long totalSize = jobConf.getLong(GRIDMIX_DISTCACHE_BYTE_COUNT, -1);\n      // Get the path of the special file\n      String distCacheFileList = jobConf.get(GRIDMIX_DISTCACHE_FILE_LIST);\n      if (fileCount < 0 || totalSize < 0 || distCacheFileList == null) {\n        throw new RuntimeException(\"Invalid metadata: #files (\" + fileCount\n            + \"), total_size (\" + totalSize + \"), filelisturi (\"\n            + distCacheFileList + \")\");\n      }\n\n      Path sequenceFile = new Path(distCacheFileList);\n      FileSystem fs = sequenceFile.getFileSystem(jobConf);\n      FileStatus srcst = fs.getFileStatus(sequenceFile);\n      // Consider the number of TTs * mapSlotsPerTracker as number of mappers.\n      int numMapSlotsPerTracker = jobConf.getInt(TTConfig.TT_MAP_SLOTS, 2);\n      int numSplits = numTrackers * numMapSlotsPerTracker;\n\n      List<InputSplit> splits = new ArrayList<InputSplit>(numSplits);\n      LongWritable key = new LongWritable();\n      BytesWritable value = new BytesWritable();\n\n      // Average size of data to be generated by each map task\n      final long targetSize = Math.max(totalSize / numSplits,\n                                DistributedCacheEmulator.AVG_BYTES_PER_MAP);","sourceCodeStart":185,"sourceCodeEnd":221,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/GenerateDistCacheData.java#L185-L221","documentation":"Error \"Invalid metadata: #files ({}), total_size ({}), filelisturi ({})\" thrown in apache/hadoop.","triggerScenarios":"The distributed-cache generation metadata (number of files, total size, or file list URI) passed to Gridmix GenerateDistCacheData is invalid or incomplete.","commonSituations":"See trigger scenarios.","solutions":["Provide consistent distcache metadata: valid file count, total size, and a file list URI."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}