- 15 May, 2024 2 commits
-
-
Younes Belkada authored
* add method * change method name * more comments * Apply suggestions from code review Co-authored-by:
Marc Sun <57196510+SunMarc@users.noreply.github.com> Co-authored-by:
amyeroberts <22614925+amyeroberts@users.noreply.github.com> * fixup * add docstrings and fix comment * warn users on the de-quantized dtype * Update src/transformers/quantizers/base.py Co-authored-by:
-
Lysandre Debut authored
* Adds support for loading GGUF files Co-authored-by:
Younes Belkada <younesbelkada@gmail.com> Co-authored-by:
99991 <99991@users.noreply.github.com> * add q2_k q3_k q5_k support from @99991 * fix tests * Update doc * Style * Docs * fix CI * Update docs/source/en/gguf.md * Update docs/source/en/gguf.md * Compute merges * change logic * add comment for clarity * add comment for clarity * Update src/transformers/models/auto/tokenization_auto.py Co-authored-by:
Younes Belkada <49240599+younesbelkada@users.noreply.github.com> Co-authored-by:
-
- 13 May, 2024 1 commit
-
-
Marc Sun authored
* fix awq test * style * add log * new fix * style * only modifying impacted model in the end * rename function
-
- 06 May, 2024 1 commit
-
-
Younes Belkada authored
Update test_hqq.py
-
- 02 May, 2024 1 commit
-
-
mobicham authored
* update HQQ transformers integration * push import_utils.py * add force_hooks check in modeling_utils.py * fix | with Optional * force bias as param * check bias is Tensor * force forward for multi-gpu * review fixes pass * remove torch grad() * if any key in linear_tags fix * add cpu/disk check * isinstance return * add multigpu test + refactor tests * clean hqq_utils imports in hqq.py * clean hqq_utils imports in quantizer_hqq.py * delete hqq_utils.py * Delete src/transformers/utils/hqq_utils.py * ruff init * remove torch.float16 from __init__ in test * refactor test * isinstance -> type in quantizer_hqq.py * cpu/disk device_map check in quantizer_hqq.py * remove type(module) nn.linear check in quantizer_hqq.py * add BaseQuantizeConfig import inside HqqConfig init * remove hqq import in hqq.py * remove accelerate import from test_hqq.py * quant config.py doc update * add hqqconfig to main_classes doc * make style * __init__ fix * ruff __init__ * skip_modules list * hqqconfig format fix * hqqconfig doc fix * hqqconfig doc fix * hqqconfig doc fix * hqqconfig doc fix * hqqconfig doc fix * hqqconfig doc fix * hqqconfig doc fix * hqqconfig doc fix * hqqconfig doc fix * test_hqq.py remove mistral comment * remove self.using_multi_gpu is False * torch_dtype default val set and logger.info * hqq.py isinstance fix * remove torch=None * torch_device test_hqq * rename test_hqq * MODEL_ID in test_hqq * quantizer_hqq setattr fix * quantizer_hqq typo fix * imports quantizer_hqq.py * isinstance quantizer_hqq * hqq_layer.bias reformat quantizer_hqq * Step 2 as comment in quantizer_hqq * prepare_for_hqq_linear() comment * keep_in_fp32_modules fix * HqqHfQuantizer reformat * quantization.md hqqconfig * quantization.md model example reformat * quantization.md # space * quantization.md space }) * quantization.md space }) * quantization_config fix doc Co-authored-by:
-
- 30 Apr, 2024 1 commit
-
-
Joao Gante authored
-
- 22 Apr, 2024 1 commit
-
-
zhong zhuang authored
* [FEAT]: EETQ quantizer support * Update quantization.md * Update docs/source/en/main_classes/quantization.md Co-authored-by:
Marc Sun <marc@huggingface.co> Co-authored-by:
-
- 17 Apr, 2024 1 commit
-
-
Younes Belkada authored
fix filing exllama test
-
- 09 Apr, 2024 1 commit
-
-
Marc Sun authored
* revert back to torch 2.1.1 * run test * switch to torch 2.2.1 * udapte dockerfile * fix awq tests * fix test * run quanto tests * update tests * split quantization tests * fix * fix again * final fix * fix report artifact * build docker again * Revert "build docker again" This reverts commit 399a5f9d9308da071d79034f238c719de0f3532e. * debug * revert * style * new notification system * testing notfication * rebuild docker * fix_prev_ci_results * typo * remove warning * fix typo * fix artifact name * debug * issue fixed * debug again * fix * fix time * test notif with faling test * typo * issues again * final fix ? * run all quantization tests again * remove name to clear space * revert modfiication done on workflow * fix * build docker * build only quant docker * fix quantization ci * fix * fix report * better quantization_matrix * add print * revert to the basic one
-
- 05 Apr, 2024 1 commit
-
-
Marc Sun authored
fix bnb test
-
- 15 Mar, 2024 1 commit
-
-
Marc Sun authored
* start integration * fix * add and debug tests * update tests * make pytorch serialization works * compatible with device_map and offload * fix tests * make style * add ref * guard against safetensors * add float8 and style * fix is_serializable * Fix shard_checkpoint compatibility with quanto * more tests * docs * adjust memory * better * style * pass tests * Update src/transformers/modeling_utils.py Co-authored-by:
David Corvoysier <david.corvoysier@gmail.com> * better comment * Update tests/quantization/quanto_integration/test_quanto.py Co-authored-by:
Arthur <48595927+ArthurZucker@users.noreply.github.com> * pass weights_only_kwarg instead * fix shard_checkpoint loading * simplify update_missing_keys * Update tests/quantization/quanto_integration/test_quanto.py Co-authored-by:
-
- 05 Mar, 2024 1 commit
-
-
Ilyas Moutawwakil authored
* added exllama kernels support for awq models * doc * style * Update src/transformers/modeling_utils.py Co-authored-by:
-
- 27 Feb, 2024 1 commit
-
-
Andrei Panferov authored
Cleaner Cache `dtype` and `device` extraction for CUDA graph generation for quantizers compatibility (#29079) * input_layernorm as the beacon of hope * cleaner dtype extraction * AQLM + CUDA graph test * is available check * shorter text test
-
- 20 Feb, 2024 1 commit
-
-
Younes Belkada authored
* forgot to push the changes for 4bit .. * trigger CI
-
- 19 Feb, 2024 2 commits
-
-
Titus authored
* generated text on A10G * generated text in CI * Apply suggestions from code review add explanatory comments Co-authored-by:
-
Younes Belkada authored
Update test_mixed_int8.py
-
- 16 Feb, 2024 1 commit
-
-
Lysandre Debut authored
* Script & Manual edition * Update
-
- 14 Feb, 2024 1 commit
-
-
Andrei Panferov authored
* aqlm init * calibration and dtypes * docs * Readme update * is_aqlm_available * Simpler link in docs * Test TODO real reference * init _import_structure fix * AqlmConfig autodoc * integration aqlm * integrations in tests * docstring fix * legacy typing * Less typings * More kernels information * Performance -> Accuracy * correct tests * remoced multi-gpu test * Update docs/source/en/quantization.md Co-authored-by:
Andrei Panferov <blacksamorez@yandex-team.ru> Co-authored-by:
-
- 12 Feb, 2024 1 commit
-
-
Klaus Hipp authored
Add language identifiers to code blocks
-
- 30 Jan, 2024 2 commits
-
-
Poedator authored
* squashed earlier commits for easier rebase * rm rebase leftovers * 4bit save enabled @quantizers * TMP gptq test use exllama * fix AwqConfigTest::test_wrong_backend for A100 * quantizers AWQ fixes * _load_pretrained_model low_cpu_mem_usage branch * quantizers style * remove require_low_cpu_mem_usage attr * rm dtype arg from process_model_before_weight_loading * rm config_origin from Q-config * rm inspect from q_config * fixed docstrings in QuantizationConfigParser * logger.warning fix * mv is_loaded_in_4(8)bit to BnbHFQuantizer * is_accelerate_available error msg fix in quantizer * split is_model_trainable in bnb quantizer class * rm llm_int8_skip_modules as separate var in Q * Q rm todo * fwd ref to HFQuantizer in type hint * rm note re optimum.gptq.GPTQQuantizer * quantization_config in __init__ simplified * replaced NonImplemented with create_quantized_param * rm load_in_4/8_bit deprecation warning * QuantizationConfigParser refactoring * awq-related minor changes * awq-related changes * awq config.modules_to_not_convert * raise error if no q-method in q-config in args * minor cleanup * awq quantizer docstring * combine common parts in bnb process_model_before_weight_loading * revert test_gptq * .process_model_ cleanup * restore dict config warning * removed typevars in quantizers.py * cleanup post-rebase 16 jan * QuantizationConfigParser classmethod refactor * rework of handling of unexpected aux elements of bnb weights * moved q-related stuff from save_pretrained to quantizers * refactor v1 * more changes * fix some tests * remove it from main init * ooops * Apply suggestions from code review Co-authored-by:
Steven Liu <59462357+stevhliu@users.noreply.github.com> Co-authored-by:
-
Omar Sanseviero authored
* Update quantization_config.py * Style * Protect from setting directly * add tests * Update tests/quantization/bnb/test_4bit.py Co-authored-by:
-
- 15 Jan, 2024 1 commit
-
-
Marc Sun authored
* fix test * reduce length * smaller model
-
- 12 Jan, 2024 2 commits
-
-
Younes Belkada authored
* add mixtral fused modules * add changes from modeling utils * add test * fix test + rope theta issue * Update src/transformers/modeling_utils.py Co-authored-by:
-
Younes Belkada authored
* add llava + fused modules * Update src/transformers/models/llava/modeling_llava.py Co-authored-by:
-
- 25 Dec, 2023 1 commit
-
-
Younes Belkada authored
* v1 * add docstring * add tests * add awq 0.1.8 * oops * fix test
-
- 21 Dec, 2023 1 commit
-
-
Poedator authored
* updated bitsandbytes.py * rm test_raise_* from test_4bit.py * add test_4bit_serialization.py * modeling_utils bulk edits * bnb_ver 0.41.3 in integrations/bitsandbytes.py Co-authored-by:
-
- 05 Dec, 2023 1 commit
-
-
Younes Belkada authored
* v1 fusing modules * add fused mlp support * up * fix CI * block save_pretrained * fixup * small fix * add new condition * add v1 docs * add some comments * style * fix nit * adapt from suggestion * add check * change arg names * change variables name * Update src/transformers/integrations/awq.py Co-authored-by:
-
- 13 Nov, 2023 1 commit
-
-
Younes Belkada authored
addresses todo for awq tests
-
- 10 Nov, 2023 1 commit
-
-
Younes Belkada authored
* add str to enum conversion * fixup * Apply suggestions from code review Co-authored-by:
-
- 02 Nov, 2023 1 commit
-
-
Younes Belkada authored
* fix for 8bit serialization * added regression tests. * fixup
-
- 01 Nov, 2023 2 commits
-
-
Marc Sun authored
* add_ xllamav2 arg * add test * style * add check * add doc * replace by use_exllama_v2 * fix tests * fix doc * style * better condition * fix logic * add deprecate msg * deprecate exllama * remove disable_exllama from the linter * remove * fix warning * Revert the commits deprecating exllama * deprecate disable_exllama for use_exllama * fix * fix loading attribute * better handling of args * remove disable_exllama from init and linter * Apply suggestions from code review Co-authored-by:
-
Younes Belkada authored
* working v1 * oops * Update src/transformers/modeling_utils.py Co-authored-by:
-
- 31 Oct, 2023 1 commit
-
-
Younes Belkada authored
fix bnb mpt test
-
- 30 Oct, 2023 1 commit
-
-
Younes Belkada authored
* fix bnb test * link to GH issue
-
- 27 Oct, 2023 1 commit
-
-
- 26 Oct, 2023 1 commit
-
-
Marc Sun authored
* add_ xllamav2 arg * add test * style * add check * add doc * replace by use_exllama_v2 * fix tests * fix doc * style * better condition * fix logic * add deprecate msg
-
- 16 Oct, 2023 1 commit
-
-
Younes Belkada authored
* First step * fix * add adjustements for gptq * change to `_pre_quantization_dtype` * Update src/transformers/modeling_utils.py Co-authored-by:
-
- 12 Oct, 2023 1 commit
-
-
Heinz-Alexander Fuetterer authored
-
- 03 Oct, 2023 1 commit
-
-
Younes Belkada authored
* fix issues with PEFT * logger warning futurewarning issues * fixup * adapt from suggestions * oops * rm test
-
- 02 Oct, 2023 1 commit
-
-
Younes Belkada authored
* fix bnb test with code revision * fix test * Apply suggestions from code review * Update src/transformers/models/auto/auto_factory.py * Update src/transformers/models/auto/auto_factory.py * Update src/transformers/models/auto/auto_factory.py
-
