Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
llama3_gptq.yaml 311 Bytes
Newer Older
chenych's avatar
update to v0.8.3  
chenych committed
1
2
3 
### model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
template: llama3
luopl's avatar
update to 0.9.2.dev0  
luopl committed
4 
trust_remote_code: true
chenych's avatar
update to v0.8.3  
chenych committed
5
6 

### export
chenych's avatar
Add QWQ-32B  
chenych committed
7 
export_dir: output/llama3_gptq
chenych's avatar
update to v0.8.3  
chenych committed
8 
export_quantization_bit: 4
chenych's avatar
Update 0429  
chenych committed
9 
export_quantization_dataset: data/c4_demo.jsonl
chenych's avatar
Add QWQ-32B  
chenych committed
10 
export_size: 5
chenych's avatar
Update 0429  
chenych committed
11 
export_device: cpu  # choices: [cpu, auto]
chenych's avatar
update to v0.8.3  
chenych committed
12 
export_legacy_format: false