# check we have at least k responses per doc, else we can't take the first k
# check we have at least k responses per doc, else we can't take the first k
assertlen(resps[0])>=self.k,f"Need at least {self.k} responses per doc to take first {self.k}, but got {len(resps[0])} only! Please increase TaskConfig.repeats ."
assert(
returnmap(lambdar:r[:self.k],resps)
len(resps[0])>=self.k
),f"Need at least {self.k} responses per doc to take first {self.k}, but got {len(resps[0])} only! Please increase TaskConfig.repeats ."
returnmap(lambdar:r[:self.k],resps)
classMajorityVoteFilter(Filter):
classMajorityVoteFilter(Filter):
...
@@ -40,6 +42,7 @@ class MajorityVoteFilter(Filter):
...
@@ -40,6 +42,7 @@ class MajorityVoteFilter(Filter):
Each entry of `resps` is a list of model responses.
Each entry of `resps` is a list of model responses.
We select the response that occurs most frequently in each entry of `resps`.
We select the response that occurs most frequently in each entry of `resps`.