Skip to content

GitLab

"vllm/inference_test.py" did not exist on "d3dd8642b1253b088946952179fe6efa9ae6125a"
Switch branch/tag
Find file Normal viewHistoryPermalink
pretrain_dataset.yaml 366 Bytes
Newer Older
jerrrrry's avatar
“13.0”  
jerrrrry committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15 
__module__: megatron.energon
__class__: Metadataset
splits:
  train:
    datasets:
      - weight: 1.
        path: <path_to_pretraining_dataset_in_energon_format>
        subflavors:
          augmentation: false
  val:
    datasets:
      - weight: 1.
        path: <path_to_pretraining_dataset_in_energon_format>
        subflavors:
          augmentation: false