Commit 36913edb authored by Jiaming Tang's avatar Jiaming Tang
Browse files

[Minor] add comments

parent bef5dd2d
......@@ -235,11 +235,12 @@ def auto_scale_block(module, module_kwargs,
module2inspect=module, kwargs=module_kwargs,
))
# attn out
# Please refer to https://github.com/mit-han-lab/llm-awq/issues/2#issuecomment-1606297469
"""
scales_list.append(_auto_get_scale(
prev_op=module.self_attention.query_key_value,
layers=[module.self_attention.dense],
inp=input_feat['self_attn.dense'],
inp=input_feat['self_attention.dense'],
))
"""
# fc1
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment