Commit f9cc0267 authored by Leo Gao's avatar Leo Gao
Browse files

Use hashed version stability test instead

parent 10d4b64a
2888d6d098a5ef8c1e7f0d8295ba80826e2e04e431f57508dfb71d53e1cd4604
\ No newline at end of file
[["Question: What is 45809 minus 41446?\nAnswer:", " 4363"], ["Question: What is 16424 minus 79811?\nAnswer:", " -63387"], ["Question: What is 76283 minus 1280?\nAnswer:", " 75003"], ["Question: What is 95869 minus 98912?\nAnswer:", " -3043"], ["Question: What is 33996 minus 38309?\nAnswer:", " -4313"], ["Question: What is 33699 minus 68525?\nAnswer:", " -34826"], ["Question: What is 89794 minus 83303?\nAnswer:", " 6491"], ["Question: What is 30221 minus 53512?\nAnswer:", " -23291"], ["Question: What is 50862 minus 28796?\nAnswer:", " 22066"], ["Question: What is 94282 minus 25820?\nAnswer:", " 68462"]]
\ No newline at end of file
de5aa6f77a2e0fd050b9c272f10c4d5d5581e4f75ffa60926f79e60ae1738960
\ No newline at end of file
[["NCIS: New Orleans (season 4) -- The fourth season of NCIS: New Orleans premiered on September 26, 2017 on CBS. The series continues to air following Bull, Tuesday at 10:00 p.m. (ET) and contained 24 episodes. The season concluded on May 15, 2018.\nQuestion: is ncis new orleans over for the season\nAnswer:", " yes"], ["NCIS: New Orleans (season 4) -- The fourth season of NCIS: New Orleans premiered on September 26, 2017 on CBS. The series continues to air following Bull, Tuesday at 10:00 p.m. (ET) and contained 24 episodes. The season concluded on May 15, 2018.\nQuestion: is ncis new orleans over for the season\nAnswer:", " no"], ["Facial hair in the military -- All branches of the U.S. Military currently prohibit beards for a vast majority of recruits, although some mustaches are still allowed, based on policies that were initiated during the period of World War I.\nQuestion: can i have a beard in the military\nAnswer:", " yes"], ["Facial hair in the military -- All branches of the U.S. Military currently prohibit beards for a vast majority of recruits, although some mustaches are still allowed, based on policies that were initiated during the period of World War I.\nQuestion: can i have a beard in the military\nAnswer:", " no"], ["Scrum (rugby) -- While the Laws of the Game continue to provide for competitive scrums, a convention exists that some scrum rules are not enforced. During the 1970s, scrum penalties for feeding the ball into the legs of the second row, packs moving off the ``mark'' or collapsing the scrum were seen as unattractive. The ability of teams to win a game purely on goals from scrum penalties was also seen as unfair. In an effort to improve this situation, changes to rules and their enforcement were made. The number of scrums was reduced with the introduction of the ``handover'' after a team has used a set of six tackles, the differential penalty, one which cannot be kicked at goal was brought in for offences at scrums and referees ceased enforcing some rules regarding feeding the ball into scrum. Aided by this change, it is common for professional teams not to fully contest scrums, according to their choice of tactics.\nQuestion: can you contest a scrum in rugby league\nAnswer:", " yes"], ["Scrum (rugby) -- While the Laws of the Game continue to provide for competitive scrums, a convention exists that some scrum rules are not enforced. During the 1970s, scrum penalties for feeding the ball into the legs of the second row, packs moving off the ``mark'' or collapsing the scrum were seen as unattractive. The ability of teams to win a game purely on goals from scrum penalties was also seen as unfair. In an effort to improve this situation, changes to rules and their enforcement were made. The number of scrums was reduced with the introduction of the ``handover'' after a team has used a set of six tackles, the differential penalty, one which cannot be kicked at goal was brought in for offences at scrums and referees ceased enforcing some rules regarding feeding the ball into scrum. Aided by this change, it is common for professional teams not to fully contest scrums, according to their choice of tactics.\nQuestion: can you contest a scrum in rugby league\nAnswer:", " no"], ["Now You See Me (film series) -- Now You See Me is a series of heist thriller films written by Ed Solomon, Boaz Yakin, and Edward Ricourt. They focus on the actions of a team of illusionists named ``The Four Horsemen'' who pull off near impossible heists. The series features an ensemble cast including Jesse Eisenberg, Mark Ruffalo, Woody Harrelson, Isla Fisher, Dave Franco, Michael Caine, Lizzy Caplan, and Morgan Freeman. The first film was released in 2013, while the second was released in 2016, and a third film is currently in development and set to be released in 2019. The series has received mixed reviews from critics and audiences and grossed nearly $700 million worldwide.\nQuestion: will there be another now you see me movie\nAnswer:", " yes"], ["Now You See Me (film series) -- Now You See Me is a series of heist thriller films written by Ed Solomon, Boaz Yakin, and Edward Ricourt. They focus on the actions of a team of illusionists named ``The Four Horsemen'' who pull off near impossible heists. The series features an ensemble cast including Jesse Eisenberg, Mark Ruffalo, Woody Harrelson, Isla Fisher, Dave Franco, Michael Caine, Lizzy Caplan, and Morgan Freeman. The first film was released in 2013, while the second was released in 2016, and a third film is currently in development and set to be released in 2019. The series has received mixed reviews from critics and audiences and grossed nearly $700 million worldwide.\nQuestion: will there be another now you see me movie\nAnswer:", " no"], ["Acute and obtuse triangles -- An acute triangle is a triangle with all three angles acute (less than 90\u00b0). An obtuse triangle is one with one obtuse angle (greater than 90\u00b0) and two acute angles. Since a triangle's angles must sum to 180\u00b0, no triangle can have more than one obtuse angle.\nQuestion: is it possible to have a triangle with two obtuse angles\nAnswer:", " yes"], ["Acute and obtuse triangles -- An acute triangle is a triangle with all three angles acute (less than 90\u00b0). An obtuse triangle is one with one obtuse angle (greater than 90\u00b0) and two acute angles. Since a triangle's angles must sum to 180\u00b0, no triangle can have more than one obtuse angle.\nQuestion: is it possible to have a triangle with two obtuse angles\nAnswer:", " no"], ["St George's Chapel, Windsor Castle -- St George's Chapel at Windsor Castle in England, is a chapel designed in the high-medieval Gothic style. It is both a Royal Peculiar, a church under the direct jurisdiction of the monarch, and the Chapel of the Order of the Garter. Seating approximately 800, it is located in the Lower Ward of the castle.\nQuestion: is st george's chapel in westminster abbey\nAnswer:", " yes"], ["St George's Chapel, Windsor Castle -- St George's Chapel at Windsor Castle in England, is a chapel designed in the high-medieval Gothic style. It is both a Royal Peculiar, a church under the direct jurisdiction of the monarch, and the Chapel of the Order of the Garter. Seating approximately 800, it is located in the Lower Ward of the castle.\nQuestion: is st george's chapel in westminster abbey\nAnswer:", " no"], ["San Juan, Puerto Rico -- San Juan, as a settlement of the Spanish Empire, was used by merchant and military ships traveling from Spain as the first stopover in the Americas. Because of its prominence in the Caribbean, a network of fortifications was built to protect the transports of gold and silver from the New World to Europe. Because of the rich cargoes, San Juan became a target of the foreign powers of the time.\nQuestion: is san juan puerto rico in the caribbean\nAnswer:", " yes"], ["San Juan, Puerto Rico -- San Juan, as a settlement of the Spanish Empire, was used by merchant and military ships traveling from Spain as the first stopover in the Americas. Because of its prominence in the Caribbean, a network of fortifications was built to protect the transports of gold and silver from the New World to Europe. Because of the rich cargoes, San Juan became a target of the foreign powers of the time.\nQuestion: is san juan puerto rico in the caribbean\nAnswer:", " no"], ["Toyota Highlander -- Announced in April 2000 at the New York Auto Show and arriving in late 2000 in Japan and January 2001 in North America, the Highlander became one of the first car-based mid-size SUV or mid-size crossovers. The Highlander is the crossover counterpart to the more rugged, truck-based midsize 4Runner and became Toyota's best-selling SUV before being surpassed by the smaller RAV4 in 2006. In Japan, the Kluger is exclusive to dealership network called Toyota NETZ as a larger alternative to the RAV4.\nQuestion: is the toyota highlander on a truck frame\nAnswer:", " yes"], ["Toyota Highlander -- Announced in April 2000 at the New York Auto Show and arriving in late 2000 in Japan and January 2001 in North America, the Highlander became one of the first car-based mid-size SUV or mid-size crossovers. The Highlander is the crossover counterpart to the more rugged, truck-based midsize 4Runner and became Toyota's best-selling SUV before being surpassed by the smaller RAV4 in 2006. In Japan, the Kluger is exclusive to dealership network called Toyota NETZ as a larger alternative to the RAV4.\nQuestion: is the toyota highlander on a truck frame\nAnswer:", " no"], ["School integration in the United States -- Some schools around America were integrated before the mid-20th century, the first ever school being Lowell High School in Massachusetts, which has accepted students of all races at its inception. The earliest known African American student, Caroline Van Vronker, attended the school in 1843. The integration of all American schools was a major catalyst for the civil rights action and racial violence that occurred in the United States during the latter half of the 20th century.\nQuestion: was integration the rule in the northern states\nAnswer:", " yes"], ["School integration in the United States -- Some schools around America were integrated before the mid-20th century, the first ever school being Lowell High School in Massachusetts, which has accepted students of all races at its inception. The earliest known African American student, Caroline Van Vronker, attended the school in 1843. The integration of all American schools was a major catalyst for the civil rights action and racial violence that occurred in the United States during the latter half of the 20th century.\nQuestion: was integration the rule in the northern states\nAnswer:", " no"], ["Rhombus -- Every rhombus is a parallelogram and a kite. A rhombus with right angles is a square.\nQuestion: do all rhombuses have 2 pairs of parallel sides\nAnswer:", " yes"], ["Rhombus -- Every rhombus is a parallelogram and a kite. A rhombus with right angles is a square.\nQuestion: do all rhombuses have 2 pairs of parallel sides\nAnswer:", " no"]]
\ No newline at end of file
{"results": {"boolq": {"acc": 0.4, "acc_stderr": 0.16329931618554522}}, "versions": {"boolq": 0}}
\ No newline at end of file
{"results": {"boolq": {"acc": 0.5048929663608562, "acc_stderr": 0.00874463623355505}}, "versions": {"boolq": 0}}
\ No newline at end of file
ec3b1bbb9561e39c43c6f77a23b4060b15c606141c5346e3d0791b3e92aaa5d0
\ No newline at end of file
[["Then they would awake, terrified and sweating, to find themselves in white starched linen, in a comfortable bed, in peaceful England. And all would be well. It may be said that although he survived it the siege nevertheless had a bad effect on the Collector.\nQuestion: the siege nevertheless had a bad effect on the Collector. True, False or Neither?\nAnswer:", " True"], ["Then they would awake, terrified and sweating, to find themselves in white starched linen, in a comfortable bed, in peaceful England. And all would be well. It may be said that although he survived it the siege nevertheless had a bad effect on the Collector.\nQuestion: the siege nevertheless had a bad effect on the Collector. True, False or Neither?\nAnswer:", " Neither"], ["Then they would awake, terrified and sweating, to find themselves in white starched linen, in a comfortable bed, in peaceful England. And all would be well. It may be said that although he survived it the siege nevertheless had a bad effect on the Collector.\nQuestion: the siege nevertheless had a bad effect on the Collector. True, False or Neither?\nAnswer:", " False"], ["A: I've never heard that one, that's very nice. Oh, so I'm all for the metric system and converting over and I think, I guess, my feeling is the way to do it is to just start giving weights, you know, have a very brief transition period and then just start giving weights and kilometers, er, just as in kilometers and weights and kilograms and everything like that and, uh, just have people start using it rather than having people constantly trying to convert. Remember me getting a package of something that said one pound, this is a package of dates mind you, was, presumably something you weigh fairly precisely, it said one pound and then in parenthesis it said four hundred fifty-four point six grams. B: Right, right. A: And, as near as I could tell, seeing that was basically anti-metric propaganda cause anyone who would say, well look I can either buy a pound of something at four hundred sixty-four point six grams which, of course, they couldn't weigh it out accurately anyway, um, every time I see something like that I think, well, that's an anti-metric argument. B: Yeah. Well, uh, I don't think it could ever happen with a quick transition.\nQuestion: it could happen with a quick transition. True, False or Neither?\nAnswer:", " True"], ["A: I've never heard that one, that's very nice. Oh, so I'm all for the metric system and converting over and I think, I guess, my feeling is the way to do it is to just start giving weights, you know, have a very brief transition period and then just start giving weights and kilometers, er, just as in kilometers and weights and kilograms and everything like that and, uh, just have people start using it rather than having people constantly trying to convert. Remember me getting a package of something that said one pound, this is a package of dates mind you, was, presumably something you weigh fairly precisely, it said one pound and then in parenthesis it said four hundred fifty-four point six grams. B: Right, right. A: And, as near as I could tell, seeing that was basically anti-metric propaganda cause anyone who would say, well look I can either buy a pound of something at four hundred sixty-four point six grams which, of course, they couldn't weigh it out accurately anyway, um, every time I see something like that I think, well, that's an anti-metric argument. B: Yeah. Well, uh, I don't think it could ever happen with a quick transition.\nQuestion: it could happen with a quick transition. True, False or Neither?\nAnswer:", " Neither"], ["A: I've never heard that one, that's very nice. Oh, so I'm all for the metric system and converting over and I think, I guess, my feeling is the way to do it is to just start giving weights, you know, have a very brief transition period and then just start giving weights and kilometers, er, just as in kilometers and weights and kilograms and everything like that and, uh, just have people start using it rather than having people constantly trying to convert. Remember me getting a package of something that said one pound, this is a package of dates mind you, was, presumably something you weigh fairly precisely, it said one pound and then in parenthesis it said four hundred fifty-four point six grams. B: Right, right. A: And, as near as I could tell, seeing that was basically anti-metric propaganda cause anyone who would say, well look I can either buy a pound of something at four hundred sixty-four point six grams which, of course, they couldn't weigh it out accurately anyway, um, every time I see something like that I think, well, that's an anti-metric argument. B: Yeah. Well, uh, I don't think it could ever happen with a quick transition.\nQuestion: it could happen with a quick transition. True, False or Neither?\nAnswer:", " False"], ["A: And you also get a lot of, uh, juries are extremely, uh, and from what I hear, I have some friends who do expert witness testimony and they say that, uh, juries are extremely vulnerable to, uh, sort of emotional pitches, you know, the prosecutor will want to, oh, I don't know show the mugging victim, you know, show the nice person he was and what a family life, and basically get the jury to be very sympathetic with the victim, or, uh, if it's a corporation, that was, uh, you know, harming some individual or something like that, they get very much, well, you know, it's just a big faceless corporation. let's make them pay as much as possible. Things like that. B: Uh-huh. A: So, not, I mean, I'm, the problem is I can't guarantee that a judge would necessarily be much better than a jury, but I'd be real nervous having a jury not at least fully agree on what the settlements would be, things like that. B: Ri-, I don't think the judge should just make the decision alone.\nQuestion: the judge should just make the decision alone. True, False or Neither?\nAnswer:", " True"], ["A: And you also get a lot of, uh, juries are extremely, uh, and from what I hear, I have some friends who do expert witness testimony and they say that, uh, juries are extremely vulnerable to, uh, sort of emotional pitches, you know, the prosecutor will want to, oh, I don't know show the mugging victim, you know, show the nice person he was and what a family life, and basically get the jury to be very sympathetic with the victim, or, uh, if it's a corporation, that was, uh, you know, harming some individual or something like that, they get very much, well, you know, it's just a big faceless corporation. let's make them pay as much as possible. Things like that. B: Uh-huh. A: So, not, I mean, I'm, the problem is I can't guarantee that a judge would necessarily be much better than a jury, but I'd be real nervous having a jury not at least fully agree on what the settlements would be, things like that. B: Ri-, I don't think the judge should just make the decision alone.\nQuestion: the judge should just make the decision alone. True, False or Neither?\nAnswer:", " Neither"], ["A: And you also get a lot of, uh, juries are extremely, uh, and from what I hear, I have some friends who do expert witness testimony and they say that, uh, juries are extremely vulnerable to, uh, sort of emotional pitches, you know, the prosecutor will want to, oh, I don't know show the mugging victim, you know, show the nice person he was and what a family life, and basically get the jury to be very sympathetic with the victim, or, uh, if it's a corporation, that was, uh, you know, harming some individual or something like that, they get very much, well, you know, it's just a big faceless corporation. let's make them pay as much as possible. Things like that. B: Uh-huh. A: So, not, I mean, I'm, the problem is I can't guarantee that a judge would necessarily be much better than a jury, but I'd be real nervous having a jury not at least fully agree on what the settlements would be, things like that. B: Ri-, I don't think the judge should just make the decision alone.\nQuestion: the judge should just make the decision alone. True, False or Neither?\nAnswer:", " False"], ["A: I'm like, I'll get a job some day and my boss will pay for it, I'll be needed. B: Yeah. A: Because, um, I didn't want to go do it myself because I didn't think I was really going to use it.\nQuestion: he was really going to use it. True, False or Neither?\nAnswer:", " True"], ["A: I'm like, I'll get a job some day and my boss will pay for it, I'll be needed. B: Yeah. A: Because, um, I didn't want to go do it myself because I didn't think I was really going to use it.\nQuestion: he was really going to use it. True, False or Neither?\nAnswer:", " Neither"], ["A: I'm like, I'll get a job some day and my boss will pay for it, I'll be needed. B: Yeah. A: Because, um, I didn't want to go do it myself because I didn't think I was really going to use it.\nQuestion: he was really going to use it. True, False or Neither?\nAnswer:", " False"], ["B: And other than that I do not think it should be allowable. I think it should be illegal for them to want to do that. it's kind of the big brother syndrome, I mean, I just, anything like that just kind of scares me. A: I tend to view it, even though I don't think I'd work for a company that did that, I sort of want to defend an employer's rights uh, in addition to an individual's rights,\nQuestion: she would work for a company that did that. True, False or Neither?\nAnswer:", " True"], ["B: And other than that I do not think it should be allowable. I think it should be illegal for them to want to do that. it's kind of the big brother syndrome, I mean, I just, anything like that just kind of scares me. A: I tend to view it, even though I don't think I'd work for a company that did that, I sort of want to defend an employer's rights uh, in addition to an individual's rights,\nQuestion: she would work for a company that did that. True, False or Neither?\nAnswer:", " Neither"], ["B: And other than that I do not think it should be allowable. I think it should be illegal for them to want to do that. it's kind of the big brother syndrome, I mean, I just, anything like that just kind of scares me. A: I tend to view it, even though I don't think I'd work for a company that did that, I sort of want to defend an employer's rights uh, in addition to an individual's rights,\nQuestion: she would work for a company that did that. True, False or Neither?\nAnswer:", " False"], ["The other set being in the pocket of Giovanna's overall from which she now drew them and held them up making it clear that they would be relinquished only upon her death and then only into the hands of Signor Kettering. What had occurred was quite contrary to the wishes of the padrone who would be outraged if he ever got to hear of it. Despite this disastrous beginning however Giovanna would be there in the morning her own family circumstances permitting and she would be much obliged if the Signora would make sure that her children were up and dressed and the breakfast eaten so that she could see that the house was returned to something like the order which Signor Kettering expected of it.\nQuestion: the house was returned to something like the order which Signor Kettering expected of it. True, False or Neither?\nAnswer:", " True"], ["The other set being in the pocket of Giovanna's overall from which she now drew them and held them up making it clear that they would be relinquished only upon her death and then only into the hands of Signor Kettering. What had occurred was quite contrary to the wishes of the padrone who would be outraged if he ever got to hear of it. Despite this disastrous beginning however Giovanna would be there in the morning her own family circumstances permitting and she would be much obliged if the Signora would make sure that her children were up and dressed and the breakfast eaten so that she could see that the house was returned to something like the order which Signor Kettering expected of it.\nQuestion: the house was returned to something like the order which Signor Kettering expected of it. True, False or Neither?\nAnswer:", " Neither"], ["The other set being in the pocket of Giovanna's overall from which she now drew them and held them up making it clear that they would be relinquished only upon her death and then only into the hands of Signor Kettering. What had occurred was quite contrary to the wishes of the padrone who would be outraged if he ever got to hear of it. Despite this disastrous beginning however Giovanna would be there in the morning her own family circumstances permitting and she would be much obliged if the Signora would make sure that her children were up and dressed and the breakfast eaten so that she could see that the house was returned to something like the order which Signor Kettering expected of it.\nQuestion: the house was returned to something like the order which Signor Kettering expected of it. True, False or Neither?\nAnswer:", " False"], ["B: and, you know, they just love kittens. A: Yeah. B: They just are fascinated. A: Oh, yeah. B: So she doesn't know that this is a cat yet.\nQuestion: this is a cat. True, False or Neither?\nAnswer:", " True"], ["B: and, you know, they just love kittens. A: Yeah. B: They just are fascinated. A: Oh, yeah. B: So she doesn't know that this is a cat yet.\nQuestion: this is a cat. True, False or Neither?\nAnswer:", " Neither"], ["B: and, you know, they just love kittens. A: Yeah. B: They just are fascinated. A: Oh, yeah. B: So she doesn't know that this is a cat yet.\nQuestion: this is a cat. True, False or Neither?\nAnswer:", " False"], ["'Very well, I'll go. But I pick my own men, and if we ever have to fight, you obey my word.'' Jehan did not think that Sidacai was in any position to impose conditions but he sat back in his chair considering.\nQuestion: Sidacai was in a position to impose conditions. True, False or Neither?\nAnswer:", " True"], ["'Very well, I'll go. But I pick my own men, and if we ever have to fight, you obey my word.'' Jehan did not think that Sidacai was in any position to impose conditions but he sat back in his chair considering.\nQuestion: Sidacai was in a position to impose conditions. True, False or Neither?\nAnswer:", " Neither"], ["'Very well, I'll go. But I pick my own men, and if we ever have to fight, you obey my word.'' Jehan did not think that Sidacai was in any position to impose conditions but he sat back in his chair considering.\nQuestion: Sidacai was in a position to impose conditions. True, False or Neither?\nAnswer:", " False"], ["B: If, uh, you've some kid uh, who's from a broken family, the schools are supposed to fill that void. It's gotten way beyond uh, you know, teaching classes and maybe providing extracurricular sports activities or something like, oh, were kind of traditionally the school's roles. A: Yeah. Yeah, it's interesting because, uh, we're just having conversation on this, uh, with a couple of people yesterday. And I was expressing my frustrations that, uh, so many problems, I work in a high school, are that kids don't have a degree of self-discipline which may be reflected in society at large. Uh, and you can't expect in a classroom for a particular course an hour a day to counteract, uh, sixteen or seventeen years of influence at home. B: Right. A: Um, and, it's seen more so because when you call parents up, many parents won't even recognize that there is a problem\nQuestion: there is a problem. True, False or Neither?\nAnswer:", " True"], ["B: If, uh, you've some kid uh, who's from a broken family, the schools are supposed to fill that void. It's gotten way beyond uh, you know, teaching classes and maybe providing extracurricular sports activities or something like, oh, were kind of traditionally the school's roles. A: Yeah. Yeah, it's interesting because, uh, we're just having conversation on this, uh, with a couple of people yesterday. And I was expressing my frustrations that, uh, so many problems, I work in a high school, are that kids don't have a degree of self-discipline which may be reflected in society at large. Uh, and you can't expect in a classroom for a particular course an hour a day to counteract, uh, sixteen or seventeen years of influence at home. B: Right. A: Um, and, it's seen more so because when you call parents up, many parents won't even recognize that there is a problem\nQuestion: there is a problem. True, False or Neither?\nAnswer:", " Neither"], ["B: If, uh, you've some kid uh, who's from a broken family, the schools are supposed to fill that void. It's gotten way beyond uh, you know, teaching classes and maybe providing extracurricular sports activities or something like, oh, were kind of traditionally the school's roles. A: Yeah. Yeah, it's interesting because, uh, we're just having conversation on this, uh, with a couple of people yesterday. And I was expressing my frustrations that, uh, so many problems, I work in a high school, are that kids don't have a degree of self-discipline which may be reflected in society at large. Uh, and you can't expect in a classroom for a particular course an hour a day to counteract, uh, sixteen or seventeen years of influence at home. B: Right. A: Um, and, it's seen more so because when you call parents up, many parents won't even recognize that there is a problem\nQuestion: there is a problem. True, False or Neither?\nAnswer:", " False"], ["During that pause I realized that Mala had not been offered a seat, nor any food or drink. The Emissary and his people were behaving as if she simply wasn't there. I could see that she was scowling and stiffening into a Mark 2 temper so I gave her an encouraging smile - which raised her as I expected to a Mark 3.\nQuestion: Mala was scowling and stiffening into a Mark 2 temper. True, False or Neither?\nAnswer:", " True"], ["During that pause I realized that Mala had not been offered a seat, nor any food or drink. The Emissary and his people were behaving as if she simply wasn't there. I could see that she was scowling and stiffening into a Mark 2 temper so I gave her an encouraging smile - which raised her as I expected to a Mark 3.\nQuestion: Mala was scowling and stiffening into a Mark 2 temper. True, False or Neither?\nAnswer:", " Neither"], ["During that pause I realized that Mala had not been offered a seat, nor any food or drink. The Emissary and his people were behaving as if she simply wasn't there. I could see that she was scowling and stiffening into a Mark 2 temper so I gave her an encouraging smile - which raised her as I expected to a Mark 3.\nQuestion: Mala was scowling and stiffening into a Mark 2 temper. True, False or Neither?\nAnswer:", " False"]]
\ No newline at end of file
{"results": {"cb": {"acc": 0.2, "acc_stderr": 0.13333333333333333, "f1": 0.14814814814814817}}, "versions": {"cb": 0}}
\ No newline at end of file
{"results": {"cb": {"acc": 0.3392857142857143, "acc_stderr": 0.06384226561930825, "f1": 0.2819143819143819}}, "versions": {"cb": 0}}
\ No newline at end of file
e8635578ed8ee70b707a666d35e468b9321db24470f80c92080651e2bfa01751
\ No newline at end of file
[["The more you would want, the less you would eat.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["The more you would want, the less you would eat.\nQuestion: Does this sentence make sense?\nAnswer:", " no"], ["John was spoke by Mary to.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["John was spoke by Mary to.\nQuestion: Does this sentence make sense?\nAnswer:", " no"], ["Paperback books lift onto the table easily.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["Paperback books lift onto the table easily.\nQuestion: Does this sentence make sense?\nAnswer:", " no"], ["She mailed John a letter, but I don't know to whom.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["She mailed John a letter, but I don't know to whom.\nQuestion: Does this sentence make sense?\nAnswer:", " no"], ["He shouted again, but I don't know who to.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["He shouted again, but I don't know who to.\nQuestion: Does this sentence make sense?\nAnswer:", " no"], ["Tom will smoke the grass, and Reuben has the hash.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["Tom will smoke the grass, and Reuben has the hash.\nQuestion: Does this sentence make sense?\nAnswer:", " no"], ["I think Agnes said that Bill would speak, but I don't remember what about.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["I think Agnes said that Bill would speak, but I don't remember what about.\nQuestion: Does this sentence make sense?\nAnswer:", " no"], ["It is obvious that Pat is lying.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["It is obvious that Pat is lying.\nQuestion: Does this sentence make sense?\nAnswer:", " no"], ["I presented Bill with it to read.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["I presented Bill with it to read.\nQuestion: Does this sentence make sense?\nAnswer:", " no"], ["Harry told Sue that Albania is a lovely place for a vacation, and Tom told Sally that Albania is a lovely place for a vacation.\nQuestion: Does this sentence make sense?\nAnswer:", " yes"], ["Harry told Sue that Albania is a lovely place for a vacation, and Tom told Sally that Albania is a lovely place for a vacation.\nQuestion: Does this sentence make sense?\nAnswer:", " no"]]
\ No newline at end of file
{"results": {"cola": {"mcc": -0.0890870806374748, "mcc_stderr": 0.22159098140155062}}, "versions": {"cola": 0}}
\ No newline at end of file
{"results": {"cola": {"mcc": -0.04538802810223175, "mcc_stderr": 0.023100371589225246}}, "versions": {"cola": 0}}
\ No newline at end of file
66276b9045b5300cba4b81340db06f674f031fa0b8883714ad0d03be464cd799
\ No newline at end of file
[["The pair of students came under scrutiny by the teacher because", " the students both received excellent grades."], ["The pair of students came under scrutiny by the teacher because", " their responses on the assignment were identical."], ["I twisted the cap off the soda bottle therefore", " the soda fizzed."], ["I twisted the cap off the soda bottle therefore", " the soda leaked out."], ["I spent the day at the pool therefore", " i sprained my ankle."], ["I spent the day at the pool therefore", " my face got sunburned."], ["The man drank heavily at the party therefore", " he had a headache the next day."], ["The man drank heavily at the party therefore", " he had a runny nose the next day."], ["The man perceived that the woman looked different because", " the woman got her hair cut."], ["The man perceived that the woman looked different because", " the woman wore a bracelet."], ["My skin broke out into a rash because", " i brushed against poison ivy in my yard."], ["My skin broke out into a rash because", " i eradicated the poison ivy from my yard."], ["The girl found a bug in her cereal therefore", " she poured milk in the bowl."], ["The girl found a bug in her cereal therefore", " she lost her appetite."], ["The woman asked the man to leave because", " he insulted her."], ["The woman asked the man to leave because", " he thanked her."], ["The judge pounded the gavel because", " the courtroom broke into uproar."], ["The judge pounded the gavel because", " the jury announced its verdict."], ["I stayed up late therefore", " i had vivid dreams that night."], ["I stayed up late therefore", " i was tired in the morning."]]
\ No newline at end of file
{"results": {"copa": {"acc": 0.5, "acc_stderr": 0.16666666666666666}}, "versions": {"copa": 0}}
\ No newline at end of file
{"results": {"copa": {"acc": 0.48, "acc_stderr": 0.050211673156867795}}, "versions": {"copa": 0}}
\ No newline at end of file
4a8605d5deed0423ec095700251ed93325b45d320aca35d4ce1e94702094435e
\ No newline at end of file
This diff is collapsed.
{"results": {"coqa": {"f1": 0.0, "f1_stderr": 0.0, "em": 0.0, "em_stderr": 0.0}}, "versions": {"coqa": 0}}
\ No newline at end of file
{"results": {"coqa": {"em": 0.0, "em_stderr": 0.0, "f1": 0.0, "f1_stderr": 0.0}}, "versions": {"coqa": 0}}
\ No newline at end of file
eb23f7d5de7528eefd8ed5f8054c402ff947319cccfef7195995946f99389201
\ No newline at end of file
[["\ndenforbid =", ["\n"]], ["\nourismt =", ["\n"]], ["\nublestro =", ["\n"]], ["\nrtersheadqua =", ["\n"]], ["\neaseincr =", ["\n"]], ["\nsmetric =", ["\n"]], ["\ntlettbar =", ["\n"]], ["\nisheryf =", ["\n"]], ["\nationliber =", ["\n"]], ["\nameworksfr =", ["\n"]]]
\ No newline at end of file
ca566c630d8ac853d5785d4b5c40a5137172c34b48af3350e1f79e6d548b36ba
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment