support CHARM (https://github.com/opendatalab/CHARM) reasoning tasks (#1190)

* support CHARM (https://github.com/opendatalab/CHARM

) reasoning tasks

* fix lint error

* add dataset card for CHARM

* minor refactor

* add txt

---------
Co-authored-by: wujiang <wujiang@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>

configs/datasets/CHARM/few-shot-examples/Global_Time_Understanding_ZH-CoT.txt

+从上下文中推断日期。
+
+Q: 请根据题目选择正确答案。今天是1937年圣诞节前夜。请问10天前的日期是多少？选项：
+(A)1988年12月14日
+(B)2026年12月14日
+(C)1950年12月14日
+(D)1938年07月14日
+(E)1937年12月14日
+(F)2007年12月14日
+A: 让我们一步一步来思考。
+如果今天是1937年的圣诞节前夕，那么今天的日期就是1937年12月24日。今天之前的10天是1937年12月14日，也就是12/14/1937。所以答案是(E)。
+
+Q: 请根据题目选择正确答案。明天是2019年11月12日。请问一年前的日期是多少？选项：
+(A)2018年08月25日
+(B)2018年11月04日
+(C)2018年11月02日
+(D)2018年11月11日
+(E)2018年09月04日
+A: 让我们一步一步来思考。
+如果明天是2019年11月12日，那么今天就是2019年11月11日。从今天起算去年的这一天是2018年11月11日。所以答案是(D)。
+
+Q: 请根据题目选择正确答案。李静和王伟在1958年1月2日结婚。今天是他们的5周年纪念日。请问明天的日期是多少？选项：
+(A)1961年01月18日
+(B)1982年01月03日
+(C)1961年01月11日
+(D)1960年10月14日
+(E)1963年01月03日
+(F)1960年12月03日
+A: 让我们一步一步来思考。
+如果李静和王伟在1958年1月2日结婚，那么如果今天是他们结婚五周年纪念日，那么今天的日期就是1963年1月2日。明天的日期是1963年1月3日，也就是1961年01月03日。所以答案是(E)。
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Chinese_Anachronisms_Judgment_Translate-EN.txt

+Identify whether a given statement contains an era error.
+
+Q: Choose the correct option if the following statement contains an anachronism. How would a person with compulsory education and basic common sense choose?
+Li Bai wrote poetry with a fountain pen.
+Options:
+(A) Yes
+(B) No
+A: Let's think step by step.
+The statement mentions "Li Bai", a Chinese poet from the Tang Dynasty. The "fountain pen" mentioned in the statement is a modern device, so Li Bai could not have used a fountain pen to write his poems, and the statement contains an anachronism. The answer is (A).
+
+Q: Choose the correct option if the following statement contains an anachronism. What would a person with compulsory education and basic common sense choose?
+During the Tang Dynasty, people could buy Dream of Red Mansions on the streets.
+Options:
+(A) Yes
+(B) No
+A: Let's think step by step.
+This statement mentions Dream of Red Mansions, which was written by Cao Xueqin, a writer of the Qing Dynasty, and there was no Dream of Red Mansions during the Tang Dynasty, so the statement contains a period error. So the answer is (A).
+
+Q: Whether the following statement contains an epochal error, choose the correct option. What would a person with compulsory education and basic common sense choose?
+On October 1, 1998, people held the five-star red flag in their hands and celebrated the National Day in Tiananmen Square
+Options:
+(A) Yes
+(B) No
+A: Let's think step by step.
+The statement mentions the year 1998, New China was founded in 1949 and the five-star red flag was established as the national flag of China in 1949, so it is reasonable to expect that October 1, 1998 was a National Day and people celebrated the National Day in Tiananmen Square, so the statement does not contain an anachronism. So the answer is (B).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Chinese_Movie_and_Music_Recommendation_Translate-EN.txt

+Given based on the given list of artworks, identify the most similar.
+
+Q: The movies that have something in common with these movies Crazy Aliens, Bullitt, Killjoys and Crazy Stone are:
+Options.
+(A) 'Lost in Thailand'
+(B)The Young and the Restless
+(C)The Children of the River and the Lake
+(D)The Mekong Operation
+A: Let's think step by step.
+Crazy Aliens is a comedy sci-fi film directed by Ning Hao, written by Liu Cixin and Sun Xiaohang, and starring Huang Bo, Shen Teng and Xu Zheng, released in 2019.
+Bullfight is a black comedy film directed by Guan Hu and starring Huang Bo and Yan Ni, released in 2009.
+Killing Life is an absurd suspense comedy film directed by Guan Hu and co-starring Huang Bo, Ren Dahua, Su Youpeng and Yu Nan, released in 2012.
+Crazy Stone is a black comedy film directed by Ning Hao and starring Guo Tao, Liu Hua, Lian Jin, Liu Gang, Xu Zheng and Huang Bo, released in 2006.
+These are all famous classic Chinese comedy movies with Huang Bo in them, and out of all the options, the only movie that seems to have something in common with these movies is Lost in Thailand (a comedy film directed by Xu Zheng and starring Huang Bo, Xu Zheng and Wang Baoqiang, released in 2012). So the answer is (A).
+
+Q: The movie that has something in common with these movies 'Red High Beam', 'Alive', 'Big Red Lantern' and 'Hero' is:
+Option.
+(A) 'Not One Less'
+(B)Let the Bullets Fly
+(C)The Legend of Ah Fei
+(D)East is East, West is West
+A: Let's think step by step.
+Red Sorghum, a war literary film directed by Zhang Yimou and starring Jiang Wen, Gong Li and Teng Rujun, was released in China in 1987.
+Alive, a drama film directed by Zhang Yimou and starring Ge You and Gong Li, was released in China in 1994.
+Hanging the Red Lantern High is a drama film directed by Zhang Yimou and starring Gong Li, He Saifei, Ma Jingwu, Cao CuiFen, Kong Lin, and Jin Shuyuan, released in China in 1991.
+Hero is a martial arts film directed by Zhang Yimou and starring Jet Li, Tony Leung, Maggie Cheung, Chen Dao Ming, Zhang Ziyi and Donnie Yen, released in China in 2002.
+These are all famous classic Chinese movies directed by Zhang Yimou. Among all the choices, the only movie that seems to have something in common with these movies is Not One Less (a drama movie directed by Zhang Yimou and starring Wei Minzhi and Zhang Huike, released in 1999). So the answer is (A).
+
+Q: The songs that have something in common with these songs 'Nocturne', 'Materia Medica', 'Listen to Your Mother' and 'Seven Miles' are:
+Option.
+(A) 'Nunchaku'
+(B) 'The Young and the Restless'
+(C) 'Pomp and Circumstance'
+(D) "Three's a Crowd
+A: Let's think step by step.
+Nocturne is a song performed by Jay Chou, written by Fong, composed by Jay Chou, and arranged by Lin Maike, and included in Jay Chou's 2005 album Chopin in November
+"Materia Medica" is a song by Jay Chou, written by Fong, composed by Jay Chou, and arranged by Michael Lam, and featured on Jay Chou's 2006 album Still Vantage Point.
+Listen to Your Mother" is a song by Jay Chou, composed by Jay Chou and arranged by Lin Maike and Hong Jingyao, included in Jay Chou's 2006 album "Still Vantage".
+Seven Miles is a song performed by Jay Chou, with lyrics by Fong, music by Jay Chou, and arrangement by Chung Hsing-min, included in Jay Chou's 2004 album of the same name, Seven Miles.
+These are all famous Chinese pop songs sung by Jay Chou. Among all the choices, the only song that seems to have something in common with these songs is "Nunchaku" (sung by Jay Chou, composed by Jay Chou, lyrics by Fong Wen-shan, arranged by Chung Hsing-min, and featured in Jay Chou's 2001 album "Fantasia"). So the answer is (A).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Chinese_Natural_Language_Inference_Translate-EN.txt

+Choose the correct answer based on the relationship between the two sentences in the question.
+
+Q:Statement 1: Xiaoming has just gone ice-skating in Shichahai
+Statement 2: It's winter in Beijing
+What is the relationship between these two statements?
+(A) Implicit
+(B) Contradictory
+(C) Irrelevant
+A: Let's think step by step.
+The first sentence mentions that Xiaoming went ice-skating in Shichahai, which is usually done in winter, and Shichahai is located in Beijing, which implies the information in the second sentence, that is, it is winter at present. So the answer is (A).
+
+Q:Statement 1：Next week, a heavy rainfall will affect the whole Hebei province
+Statement 2: Next week, the weather in Shanghai will be fine.
+What is the relationship between these two statements?
+(A) Implied
+(B) Contradictory
+(C) Irrelevant
+A: Let's think step by step.
+These two sentences describe the weather in two geographical locations, one in Hebei Province and the other in Shanghai. Hebei Province and Shanghai are geographically far apart, so the weather conditions in these two places are not necessarily directly related. Therefore, the relationship between the two sentences is irrelevant. So the answer is (C).
+
+Q:Statement 1: Yesterday, Xiao Ming watched the sunset on a hill and exclaimed, "The sunset is infinite, but it's just near dusk"
+Statement 2: Yesterday it rained and Ming didn't go out
+What is the relationship between these two statements?
+(A) implied
+(B) contradictory
+(C) Irrelevant
+A: Let's think step by step.
+The first sentence says that Ming saw the sunset on the mountain yesterday, while the second sentence says that it rained yesterday and Ming didn't go out. There is a contradiction between these two sentences because if Ming didn't go out, then he couldn't have seen the sunset on the mountain. So there is a contradiction between these two sentences. So the answer is (B).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Chinese_Reading_Comprehension_Translate-EN.txt

+Please understand the meaning of the question and choose the correct answer.
+
+Q:Some Cantonese people don't like chili peppers. Therefore, some southerners don't like chili peppers. Which of the following ensures the validity of the above argument?
+(A) Some Cantonese people love chili peppers
+(B) Some Southerners love chili peppers.
+(C) All Cantonese are Southerners.
+(D) Some Cantonese people do not love chili or sweets.
+A: Let's think step by step.
+In this argument, we infer from "some Cantonese don't like chili peppers" that "some Southerners don't like chili peppers". This inference depends on the relationship between Cantonese and Southerners. In order for this inference to be valid, we need to make sure that at least some Cantonese are Southerners. Therefore, option (C) "All Cantonese are southerners" is the one that ensures that this argument is valid. So the answer is (C).
+
+
+Q:Thangka is a form of painting rich in Tibetan cultural characteristics, which has a history of more than 1300 years since the rise of the Tubo Dynasty and is a cultural treasure of the Snowy Plateau. It is a cultural treasure of the Snowy Plateau. Its subject matter is not only religious, but also historical and folklore content, so it is also known as the "encyclopedia" to understand Tibet. Therefore, if you want to understand the history of Tibet, in addition to sitting down and reading the strict but slightly dull history books, you can also choose a pleasant and enjoyable way - enjoying the thangka, conversing with many ancient people, imagining the historical events, savoring the rich spiritual world of the Tibetans, and understanding the unique folklore of the Tibetans, which is a process to make the history become a three-dimensional and palpable. This is a process of making history three-dimensional and palpable.
+(A) Thangkas can provide a cozy and relaxing way to learn about Tibet.
+(B) The thangkas are a unique record of Tibetan folklore and historical events.
+(C) The thangka is an "encyclopedic" form of painting for understanding Tibetan culture and history.
+(D) The thangka is a form of painting that is rich in Tibetan cultural characteristics and has a long history.
+A: Let's think step by step.
+It is clearly mentioned in the text that apart from reading rigorous but dull history books, appreciating thangkas is a pleasant and enjoyable way to talk with many ancient people, imagine historical events, experience the rich spiritual world of the Tibetans, and learn about the unique Tibetan folklore. So the main intention of the passage is (A) "Thangkas can provide a cozy and relaxing way to learn about Tibet." So the answer is (A).
+
+Q:As a principle and method of literary criticism, the concept of "knowing people and discussing the world" was first proposed by Mencius, a thinker of the Warring States period. According to Mencius, in order to make friends with the ancients, it is not enough just to read their poems and books, but also to understand their behavior and the times in which they lived, so as to read and understand their poems and books, and to make friends with them and become their soulmates. What is the incorrect understanding of this passage?
+(A) People's hearts can communicate and resonate with each other.
+(B) "Knowing people and discussing the world" has been used as a kind of literary criticism and has shown a strong vitality.
+(C) "Knowing people and discussing the world" can help the descendants to make friends with the ancients and the ancients to become their soulmates.
+(D) Knowing about the ancients and their times helps to understand their works.
+A: Let us think step by step
+From the passage, we fail to see that (B) "Knowing the people of the world" has been used as a kind of literary criticism and has shown a strong vitality. Although it is true that "knowing people and discussing the world" is a principle and method of literary criticism put forward by the philosopher Mencius during the Warring States period, the passage does not mention that "knowing people and discussing the world" is still in use today, nor does it mention that it has shown great vitality. Therefore, option (B) is a misconception. So the answer is (B).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Chinese_Sequence_Understanding_Translate-EN.txt

+Choose the correct answer based on the context
+
+Q: The following characters are correctly ordered in chronological order? Options:
+(A) Qin Shi Huang, Zhuge Liang, Liu Bang, Bai Juyi
+(B) Zhuge Liang, Qin Shi Huang, Liu Bang, Bai Ju Yi
+(C) Qin Shi Huang, Liu Bang, Zhu Geliang, Bai Juyi
+(D) Bai Juyi, Zhu Geliang, Liu Bang, Qin Shi Huang
+A: Let's think step by step.
+There are 4 characters mentioned in the options, in which Qin Shi Huang is from Qin Dynasty, Zhuge Liang is from Three Kingdoms period, Liu Bang is from Han Dynasty period and Bai Juyi is from Tang Dynasty period, according to the chronological order they are Qin Dynasty, Han Dynasty, Three Kingdoms period and Tang Dynasty, so the characters are sequentially Qin Shi Huang, Liu Bang, Zhuge Liang and Bai Juyi. So the answer is (C).
+
+Q:The following words describing age are correctly ordered from youngest to oldest age? Options:
+(A) Unconfused, Established, Knowledge of life, Flowering age
+(B) To stand, not to be confused, to know one's destiny, and to be in the prime of life.
+(C) Being in the prime of life, knowing one's destiny, being in the prime of life, not being confused.
+(D) to stand up, to grow old, to be unperturbed, to know one's destiny
+A: Let us think step by step.
+The options mention four words describing age. "Established" means 30 years old, "not perplexed" means 40 years old, "knowing one's destiny" means 50 years old, and "at the age of 60" means 50 years old. Therefore, the order of age from youngest to oldest is "to be established", "not to be perplexed", "to know one's life" and "to be at the end of one's life". So the answer is (B).
+
+Q:Which of the following steps is correct for making old-fashioned lollipops? Options:
+(A) Preparing the ingredients, pouring the syrup into the molds, making the syrup, cooling to set the shape
+(B) Prepare ingredients, make syrup, pour syrup into molds, cool to set
+(C) Prepare ingredients, pour syrup into mold, cool and set, make syrup
+(D) Prepare ingredients, cool and set, make syrup, pour syrup into molds
+A: Let's think step by step.
+The question refers to the steps in making old-fashioned lollipops, and the options include "prepare ingredients", "pour syrup into molds", and "make syrup", "According to the steps of making old-fashioned lollipops, the first step is to prepare the ingredients, then you need to make the syrup, then pour the syrup into the molds, and finally cool and set the lollipops. So the answer is (B).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Chinese_Sport_Understanding_Translate-EN.txt

+Determine whether a human-constructed sentence about sports is plausible.
+
+Q: Is the following sentence plausible? "Athlete Zhang Peimeng's consistent passes made the offensive end take the initiative"
+Options:
+(A) Credible
+(B) Not credible
+A: Let's think step by step.
+Zhang Peimeng is a Chinese track and field athlete, and first pass is a volleyball term. Since he is not a volleyball player, he would not be involved in the passing part of a volleyball game. So the answer is (B).
+
+Q: Is the following sentence plausible? "Athlete Guo Allen scored a free kick."
+Options:
+(A) Credible
+(B) Unbelievable
+A: Let's think step by step.
+Guo Allen is a Chinese basketball player, and free kick is a soccer term. Since he is not a soccer player, he won't be in involved in the free kick part of a soccer game, so the answer is (B).
+
+Q: Is the following sentence plausible? "The athlete Fan Zhiyi scored a goal."
+Options:
+(A) Credible
+(B) Not credible
+A: Let's think step by step.
+Fan Zhiyi is a Chinese soccer player, and scoring a goal is a soccer sport term. Since he is a professional soccer player, he is likely to score a goal, so the answer is (A).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Chinese_Time_Understanding_Translate-EN.txt

+Choose the correct answer based on the context.
+
+Q: Siu Wah was born in the year C Shen and his brother is 6 years older than him, so in which year was his brother born?
+Options:
+(A) Year of Ji Mao
+(B) The year of Geng Yin
+(C) Bingshen year
+(D) The year of Yi Chou
+A: Let's think step by step.
+Xiaohua was born in the year of Cingshen. His brother is six years older than him, and according to the Chinese Heavenly Stems and Earthly Branches dating system, the sixth year before the year of Cingshen is the year of Geng Yin. So the answer is (B).
+
+Q: If this is the year of A-chen, and Li Hua's grandfather said, "This is the year I know my destiny," what is Zhao Ting's grandfather's zodiac sign?
+Options:
+(A) Dog
+(B) Tiger
+(C) Rooster
+(D) Rat
+A: Let's think step by step.
+In the title, it is mentioned that grandpa is "in the year of knowing his destiny". In ancient China, "in the year of knowing his destiny" refers to the age of 50, so grandpa is 50 years old this year, and this year is the year of A-chen. According to the method of chronology of the Chinese Heavenly Stems and Earthly Branches, the year of grandpa's birth was the year of A-yin, and the year of A-yin is the year of the Tiger, so grandpa is a Tiger. The year he was born is the year of the tiger, so he is a tiger. So the answer is (B).
+
+Q: Li started making a drawing at 3:00 p.m. in the morning, and he knew that it would take him 10 hours to finish it. So what is the earliest hour at which he can finish it?
+Options:
+ (A) Tatsu
+ (B) U.M. hour
+ (C) Midday
+ (D) the hour of midnight
+A: Let's think step by step.
+According to the ancient Chinese method of timekeeping, three moments of the Ohio hour means approximately 21:45, and 10 hours later it is 7:45, which is the hour of the hour of the moon. So the answer is (A).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Global_Anachronisms_Judgment_Translate-EN.txt

+Identify whether a given statement contains an era error.
+
+Q: How would a person with compulsory education and basic common sense answer whether the following statement contains an anachronism?
+During the landing operations on Iwo Jima, Ralph spoke loudly into the radio.
+Options:
+(A) Yes
+(B) No
+A: Let's think step by step.
+The statement refers to "during the Iwo Jima landings," which is a historical event during World War II. At that time, it was accurate and appropriate for military personnel to use radios to communicate with each other. So the answer is (B).
+
+Q: Does the following statement contain an anachronism and how would a person with a compulsory education and basic common sense answer?
+During the landing operations on Iwo Jima, Ralph speaks loudly into his iPhone.
+Options:
+(A) Yes
+(B) No
+A: Let's think step by step.
+The statement mentions "during the Iwo Jima landings," which refers to a historical event during World War II. However, the reference to Ralph speaking loudly into his iPhone introduces an anachronism; the iPhone is a modern smartphone that did not exist during the Iwo Jima landings in 1945. So the answer is (A).
+
+Q: How would a person with compulsory education and basic common sense answer whether the following statement contains an anachronism?
+Nothing is more satisfying than Sally Hemmings knitting a sweater while watching HBO's True Detective.
+Options:
+(A) Yes
+(B) No
+A: Let's think step by step.
+This statement mentions Sally Hemings, an enslaved woman in late 18th and early 19th century America. However, the reference to her knitting while watching HBO's True Detective introduces an anachronism. Television did not exist in the time period in which Sally Hemings lived, and the specific reference to watching a particular program like True Detective is clearly inappropriate for that time period in history. So the answer is (A).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Global_Movie_and_Music_Recommendation_Translate-EN.txt

+Given based on the given list of artworks, identify the most similar.
+
+Q: Find a movie similar to Braveheart, Pretty Woman, Schindler's List, Apollo 13:
+Options:
+(A) Star Trek.
+(B) My Grand Greek Wedding 2
+(C)The Santa Clause 2
+(D) Dances with Wolves
+A: Let's think step by step.
+Star Trek is a science fiction movie, and while it has depth and complexity, its themes and style are quite different from the four movies mentioned above.
+MY GREAT GREEK WEDDING 2 is a light-hearted comedy movie that differs considerably in theme and style from the four movies mentioned above.
+Santa Claus 2 is a Christmas-themed family movie that also differs considerably in theme and style from the four movies mentioned above.
+Dances with Wolves is a movie depicting the relationship with Native Americans during the pioneering period of the American West.
+The theme and style of one of the movies, Dances with Wolves, matches the four movies mentioned above. This movie, like Braveheart, Pretty Woman, Schindler's List, and Apollo 13, is a historical movie with depth and seriousness. So the answer is (D).
+
+Q: Find a movie similar to Braveheart, Pretty Woman, Apollo 13, Dances with Wolves:
+Options:
+(A) Batman: Assault on Arkham
+(B)The Shawshank Redemption
+(C) "Toy Story
+(D) The Lion King
+A: Let's think step by step.
+Batman: Assault on Arkham is a superhero movie whose theme and style are quite different from the four movies mentioned above.
+The Shawshank Redemption is a 1994 American drama film directed by Frank Delabonte and starring Tim Robbins and Morgan Freeman; it's a movie about hope and perseverance.
+Toy Story is an animated movie, and while it may have some themes of adventure and friendship, its themes and style differ considerably from the four movies mentioned above.
+The Lion King, while it is a classic animated movie that covers themes of courage and growing up, has themes and styles that differ considerably from the four movies mentioned above.
+One of the movies, The Shawshank Redemption, like Braveheart, Pretty Woman, Apollo 13, and Dances with Wolves, is a movie with depth and seriousness and has a similar theme and style to the other three movies. So the answer is (B).
+
+Q: Find a movie similar to Astonishing, Braveheart, Lowlife, Schindler's List:
+Options:
+(A) The Cabin of Dr. Caligari
+(B)The Shawshank Redemption
+(C) 'Spider-Man 2'
+(D)Taxi
+A: Let's think step by step.
+The Cabin of Dr. Caligari is a 1920 German Expressionist silent film directed by Robert Wiene. It is often considered a landmark work of German Expressionist cinema and one of the earliest horror films.
+The Shawshank Redemption is a 1994 American drama film directed by Frank Delabonte and starring Tim Robbins and Morgan Freeman; it is a movie about hope and perseverance.
+Spider-Man 2 is a 2004 American superhero film directed by Sam Raimi and starring Tobey Maguire as the second installment of the Spider-Man trilogy.
+Cab is a 2004 American comedy-action film directed by Tim Story and starring Jimmy Foran and Quentin Latafa. The movie is an American remake of the 1998 French film.
+And the titles, "The Thing," "Braveheart," "Lowlife," and "Schindler's List" are some very deep, richly plotted movies with strong human themes. They have all won high praise from audiences and critics for their excellent scripts, brilliant performances and profound themes. Among the options is The Shawshank Redemption, which tells the story of how a wrongly accused banker keeps hope alive in prison and eventually escapes. With its deeply moving plot and profound depiction of human nature, this movie has many similarities with the movie in the title. So the answer is (B).

configs/datasets/CHARM/few-shot-examples_Translate-EN/Global_Natural_Language_Inference_Translate-EN.txt

+Choose the correct answer based on the relationship between the two sentences in the question.
+
+Q:Statement 1: But the old man carefully picked the snake away from the grass, letting it crawl towards the grass, mumbling: sin, sin, this was your home!
+Statement 2: The old man was very sad in his heart.
+What is the relationship between these two statements?
+(A) Contained
+(B) contradictory
+(C) Irrelevant
+A: Let's think step by step.
+The first sentence describes an old man carefully handling a snake's row. The old man says, "Sin, sin, sin," indicating that he feels guilty and sad because he has violated the snake's territory. The second sentence can be inferred from the fact that the old man was very sad. Therefore, the two sentences are implicitly related. So the answer is (A).
+
+Q:Statement 1: She was a black explosive-headed woman with a plump face, full lips and big teeth.
+Statement 2: She likes to wear red lipstick and red clothes.
+What is the relationship between these two statements?
+(A) implied
+(B) contradictory
+(C) Unrelated
+A: Let's think step by step.
+Both sentences describe the same woman, but they focus on different features. The first sentence describes her physical features, including her face, lips, teeth, and hair. The second sentence describes her aesthetic preferences, including lipstick color and dress color. There is no obvious implication or contradiction between these two sentences, so we can say that they are unrelated. So the answer is (C).
+
+Q:Statement 1: You're not sure you've made it clear whose side you're on.
+Statement 2: It is obvious whose side you are on.
+What is the relationship between these two statements?
+(A) Contained
+(B) Contradictory
+(C) Unrelated
+A: Let's think step by step.
+The first sentence indicates that you are not sure who you support, while the second sentence makes it clear that your position is obvious, i.e. you know exactly who you support. Therefore, the two sentences contradict each other. So the answer is (B).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Global_Reading_Comprehension_Translate-EN.txt

+Please understand the meaning of the question and choose the correct answer.
+
+Q:A person's intake of refined sugar and foods that break down into sugar during digestion is almost all of that person's dietary source of glucose (a type of sugar) in the bloodstream. However, coffee itself does not break down into sugar during digestion, which can sometimes cause a person's blood sugar levels to rise sharply, even if the coffee is not sweetened with cream or any sweetener. Which of the following, if true, would best help explain the effect of coffee on blood sugar levels as described above? Options:
+(A) People with sedentary office jobs are more likely to drink large amounts of coffee and to have higher blood sugar levels than people with jobs that require constant physical activity.
+(B) People often drink coffee with meals that consist of several foods that break down quickly into sugar during digestion.
+(C) For many people, a slice of chocolate cake consumes the same stimulating effect as a regular cup of coffee.
+(D) Drinking more than two cups of coffee in an hour increases a person's stress level, and the body responds to stress by releasing stored glucose into the bloodstream.
+A: Let's think step by step.
+The key to the question is to find a factor that explains how coffee causes blood sugar levels to rise, even though coffee itself does not break down into sugar during digestion. This requires us to find a mechanism that can explain how coffee indirectly affects blood sugar levels. Option (D) "Drinking more than two cups of coffee in an hour increases a person's stress levels, and the body responds to stress by releasing stored glucose into the bloodstream" would, if true, best help to explain the effect of coffee on blood sugar levels. This option provides a possible mechanism by which the coffee-induced stress response may cause the body to release stored glucose, leading to an increase in blood sugar levels. This explains why coffee (even without creamer or sweetener) causes blood sugar levels to rise, even though it does not break down into sugar during digestion. So the answer is (D).
+
+Q:A person has tried several different psychotherapies for a mental illness:psychoanalytic therapy,cognitive behavioral therapy and sandplay therapy. He says: "The psychotherapy process made me very unhappy and, therefore, these therapies are ineffective." Which of the following, if true, would most strongly challenge the above conclusion? Options:
+(A) Several different psychotherapies target different mental illnesses.
+(B) People who try multiple psychotherapies are happier than those who try only one.
+(C) Trying different psychotherapies at the same time makes it easier to find the one that works
+(D) People who do well in therapy tend to feel unhappy during the therapy process
+A: Let's think step by step.
+Understanding the question shows that if unhappiness during psychotherapy is a sign of a good therapeutic outcome, then the person's conclusion - that the therapy is ineffective because he feels unhappy during the process - may be wrong. He may have misinterpreted the difficulty and discomfort of the therapy process as a sign that the therapy is ineffective, when in fact it may be a sign that the therapy is working. (D) "People whose therapy is working well tend to feel unhappy during the therapy process" If true, this would most strongly challenge the above conclusion. So the answer is (D).
+
+Q:For a book to be a bestseller,it must be readable or well packaged. If the above assertion holds true, which of the following must be true? Option:
+(A) A book that is not readable and not well-packaged must not be a bestseller.
+(B) A book that is not carefully packaged must not be a bestseller.
+(C) A book that is readable must be a bestseller
+(D) Most people like bestsellers that are readable.
+A: Let's think step by step.
+According to the above assertion, "For a book to be a bestseller, it must be readable or well-packaged", so (A) "A book that is neither readable nor well-packaged must not be a bestseller" must be true. Because if a book is neither readable nor well-packaged, then it does not fulfill the conditions for being a bestseller. So the answer is (A).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Global_Sequence_Understanding_Translate-EN.txt

+Choose the correct answer based on the context
+
+Q:Which of the following lists is correct in chronological order? Options:
+(A) University, M.A., Ph.D., Post-doctoral
+(B) University, Ph.D., M.Phil, Post Ph.D.
+(C) University, High School, Post-doctoral, Doctoral
+(D) Post-doctoral, University, Master's, PhD
+A: Let's think step by step.
+The options in the question introduce the concept of four qualifications "University, Masters, PhD, Post-doctoral" and in the order in which these qualifications were obtained, the first qualification obtained is University, followed by Masters, then PhD and finally Post-doctoral. So the answer is (A).
+
+Q:Which of the following list is correct in chronological order? Options:
+(A) May 1, July 15, August 12, June 14
+(B) July 15, August 12, May 1, June 14
+(C) August 12, May 1, July 15, June 14
+(D) May 1, June 14, July 15, August 12
+A: Let's think step by step.
+The question option introduces four dates which are May 1, June 14, July 15, August 12 in chronological order of the year. So the answer is (D).
+
+Q:Which of the following lists is correct in chronological order? Options:
+(A) Countdown to Heaven, Magicians at the End of the Century, The Undead of Baker Street, Detectives' Requiem
+(B) The Magician at the End of the Century, The Undead of Baker Street, Countdown to Heaven, The Detectives' Requiem
+(C) The Magician at the End of the Century, Countdown to Heaven, The Undead of Baker Street, The Detectives' Requiem
+(D) The Magician at the End of the Century, The Undead of Baker Street, The Detectives' Requiem, The Countdown to Heaven
+A: Let's think step by step.
+The options mention the names of the four Conan theatrical releases. The Magician at the End of the Century was released in 1999, The Undead of Baker Street was released in 2002, Countdown to Heaven was released in 2001, and The Detectives' Requiem was released in 2006, so in chronological order it's The Magician at the End of the Century, Countdown to Heaven, The Undead of Baker Street, and The Detectives' Requiem. So the answer is (C).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Global_Sport_Understanding_Translate-EN.txt

+Determine whether a human-constructed sentence about sports is plausible.
+
+Q: Is the following sentence credible? "Tyson threw a right straight punch."
+Options:
+(A) Credible
+(B) Not credible
+A: Let's think step by step.
+Tyson is an American professional boxer. The right straight punch is a common move in boxing. It's quite normal for a boxer to throw a right straight punch, so this sentence is credible. The correct answer should be (A).
+
+Q: Is the following sentence credible? "Beckham completed a slam dunk."
+Options:
+(A) Credible
+(B) Not credible
+A: Let's think step by step.
+Beckham is an English professional footballer. A slam dunk is a technical move in basketball. Therefore, this sentence is not credible. The correct answer should be (B).
+
+Q: Is the following sentence credible? "LeBron James made an eagle."
+Options:
+(A) Credible
+(B) Not credible
+A: Let's think step by step.
+LeBron James is a well-known American professional basketball player. An eagle is a term in golf, meaning two strokes under par. Therefore, this sentence is not credible. The correct answer should be (B).
\ No newline at end of file

configs/datasets/CHARM/few-shot-examples_Translate-EN/Global_Time_Understanding_Translate-EN.txt

+Infer dates from context.
+
+Q: Choose the correct answer according to the question. Today is Christmas Eve 1937. What is the date 10 days ago? Options:
+(A) December 14, 1988
+(B)December 14, 2026
+(C)December 14, 1950
+(D)July 14, 1938
+(E)December 14, 1937
+(F) December 14, 2007
+A: Let's think step by step.
+If today is Christmas Eve, 1937, then today's date is December 24, 1937. The 10 days before today is December 14, 1937, which is 12/14/1937. so the answer is (E).
+
+Q: Select the correct answer as per the question. Tomorrow is 12/11/2019. What is the date one year ago? Options:
+(A) August 25, 2018
+(B)November 04, 2018
+(C)November 02, 2018
+(D)November 11, 2018
+(E) September 04, 2018
+A: Let's think step by step.
+If tomorrow is November 12, 2019, then today is November 11, 2019. This day last year from today is November 11, 2018 So the answer is (D).
+
+Q: Select the correct answer according to the question. Li Jing and Wang Wei were married on 2/1/1958. Today is their 5th anniversary. What is the date for tomorrow, please? Options:
+(A) January 18th, 1961
+(B)January 03, 1982
+(C)January 11, 1961
+(D)October 14, 1960
+(E)January 03, 1963
+(F) December 03, 1960
+A: Let's think step by step.
+If Li Jing and Wang Wei were married on January 2, 1958, then today's date would be January 2, 1963 if today is their fifth wedding anniversary. Tomorrow's date will be January 3, 1963, which is January 03, 1961. So the answer is (E).
\ No newline at end of file

configs/eval_charm.py

+from mmengine.config import read_base
+
+with read_base():
+    from .datasets.CHARM.charm_rea_gen_f8fca2 import charm_rea_datasets as datasets
+    from .models.hf_internlm.lmdeploy_internlm2_chat_7b import models as lmdeploy_7b_chat_model
+
+    # from models.openai.gpt_3_5_turbo_1106 import models as gpt_3_5_turbo_1106_model
+    # from models.openai.gpt_4_1106_preview import models as gpt_4_1106_preview_model
+
+    # from .models.chatglm.hf_chatglm3_6b_32k import models as chatglm3_6b_32k_model
+    # from .models.yi.hf_yi_6b_chat import models as yi_6b_chat_model
+    # from .models.hf_internlm.hf_internlm2_chat_7b import models as hf_internlm2_chat_7b_model
+    # from .models.deepseek.hf_deepseek_7b_chat import models as deepseek_7b_chat_model
+    # from .models.baichuan.hf_baichuan2_7b_chat import models as baichuan2_7b_chat_model  # need torch 2.1
+    # from .models.hf_llama.hf_llama2_7b_chat import models as llama2_7b_chat_model
+    # from .models.vicuna.hf_vicuna_7b_v15_16k import models as vicuna_7b_v15_16k_model
+
+    # from .models.baichuan.hf_baichuan2_13b_chat import models as baichuan2_13b_chat_model  # need torch 2.1
+    # from .models.hf_llama.hf_llama2_13b_chat import models as llama2_13b_chat_model
+    # from .models.vicuna.hf_vicuna_13b_v15_16k import models as vicuna_13b_v15_16k_model
+    # from .models.hf_internlm.hf_internlm2_chat_20b import models as hf_internlm2_chat_20b_model
+
+    # from .models.yi.hf_yi_34b_chat import models as yi_34b_chat_model
+
+    # from .models.deepseek.hf_deepseek_67b_chat import models as deepseek_67b_chat_model
+    # from .models.hf_llama.hf_llama2_70b_chat import models as llama2_70b_chat_model
+
+    # from .models.hf_llama.hf_llama3_8b_instruct import models as llama3_8b_instruct_model
+    # from .models.hf_llama.hf_llama3_70b_instruct import models as llama3_70b_instruct_model
+
+models = sum([v for k, v in locals().items() if k.endswith('_model')], [])
+work_dir = './outputs/CHARM/chat/'

opencompass/datasets/__init__.py

@@ -12,6 +12,7 @@ from .bustum import *  # noqa: F401, F403
 from .c3 import *  # noqa: F401, F403
 from .cb import *  # noqa: F401, F403
 from .ceval import *  # noqa: F401, F403
+from .charm import *  # noqa: F401, F403
 from .chembench import *  # noqa: F401, F403
 from .chid import *  # noqa: F401, F403
 from .cibench import *  # noqa: F401, F403

opencompass/datasets/charm.py

+import json
+import os.path as osp
+import re
+
+from datasets import Dataset
+
+from opencompass.openicl.icl_evaluator import BaseEvaluator
+from opencompass.registry import (ICL_EVALUATORS, LOAD_DATASET,
+                                  TEXT_POSTPROCESSORS)
+
+from .base import BaseDataset
+
+
+@TEXT_POSTPROCESSORS.register_module('charm-rea')
+def charm_rea_postprocess(text: str) -> str:
+    ans = text
+    ans_line = ans.split('answer is ')
+    if len(ans_line) != 1:
+        ans = ans_line[1].strip()
+    match = re.search(r'\(([A-Z])\)*', ans)
+    if match:
+        return match.group(1)
+    match = re.search(r'([A-Z])', ans)
+    if match:
+        return match.group(1)
+    return ans
+
+
+@ICL_EVALUATORS.register_module()
+class CharmReaEvaluator(BaseEvaluator):
+
+    def score(self, predictions, references):
+        if len(predictions) != len(references):
+            return {
+                'error': 'predictions and references have different '
+                'length'
+            }
+        details = []
+        cnt = 0
+        for pred, ref in zip(predictions, references):
+            detail = {'pred': pred, 'answer': ref, 'correct': False}
+            if pred == ref:
+                cnt += 1
+                detail['correct'] = True
+            details.append(detail)
+
+        score = cnt / len(predictions) * 100
+
+        return {'score': score, 'details': details}
+
+
+@LOAD_DATASET.register_module()
+class CharmDataset(BaseDataset):
+
+    @staticmethod
+    def load(path: str, name: str):
+        with open(osp.join(path, f'{name}.json'), 'r') as f:
+            data = json.load(f)['examples']
+        dataset = Dataset.from_list(data)
+        return dataset