Skip to content

GitLab

Unverified Commit 1cdc8861 authored by Szymon Ożóg's avatar Szymon Ożóg Committed by GitHub
Browse files

Missing comment explaining VDR variable in GGUF kernels (#13290)

parent 31aa045c
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
csrc/quantization/gguf/vecdotq.cuh
View file @ 1cdc8861
...@@ -37,6 +37,8 @@ static __device__ __forceinline__ int get_int_from_uint8_aligned(const uint8_t * ...@@ -37,6 +37,8 @@ static __device__ __forceinline__ int get_int_from_uint8_aligned(const uint8_t *
return *((const int *) (x8 + sizeof(int) * i32)); // assume at least 4 byte alignment return *((const int *) (x8 + sizeof(int) * i32)); // assume at least 4 byte alignment
} }
// VDR = vec dot ratio, how many contiguous integers each thread processes when the vec dot kernel is called
// MMVQ = mul_mat_vec_q, MMQ = mul_mat_q
#define VDR_Q4_0_Q8_1_MMVQ 2 #define VDR_Q4_0_Q8_1_MMVQ 2
#define VDR_Q4_0_Q8_1_MMQ 4 #define VDR_Q4_0_Q8_1_MMQ 4
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment