适配后端vllm

dataflow/example/AgenticRAGPipeline/eval_test_data.jsonl

+{"id": "17989685", "contents": "\"The Little Vagabond\"\nat church, Would not have bandy children, nor fasting, nor birch. And God, like a father, rejoicing to see His children as pleasant and happy as He, Would have no more quarrel with the Devil or the barrel, But kiss him, and give him both drink and apparel. The top of this plate displays a larger man, possibly God, laying over and protecting a boy. The lower section shows a group of people of different sizes hugging, possibly parents and children, sitting around a fire. The two halves of \"\"Songs of Innocence and Experience\"\" differ in more ways than the"}
+{"id": "4864947", "contents": "\"Halloween 4: The Return of Michael Myers\"\nwhen reviewing the producers did not like how the mask turned out. It was allegedly customized again but did not live up to the original, and the producers felt it was too old and went for a new mask. Some scenes had to be re-shot with the new mask. The only scene left in is when Loomis is thrown through a glass door; as Michael comes up behind him, the unaltered face and blonde hair is visible. After viewing the film's rough cut, director Dwight H. Little and producer Moustapha Akkad decided that the film's violence was too soft, and"}
+{"id": "12455656", "contents": "\"John Scale\"\na Captain in St Petersburg, in the weeks leading up to the killing, Scale is recorded as having met with Oswald Rayner and Felix Yusupov in the diary of their chauffeur, William Compton. Several other books and documentaries have claimed Scale's involvement in Rasputin's death, or even alleged that Scale commanded Rayner to fatally shoot him. A letter from Alley to Scale provides the best evidence of British Intelligence involvement in the murder and torture that reads : In March 1918, Scale was an SIS bureau commander in Stockholm. John Scale Lieutenant Colonel John Dymoke Scale DSO, OBE (born 27"}
+{"id": "18253508", "contents": "\"Trina Radke\"\nstruggles with CFIDS, entitled \"\"Be \"\"Your\"\" Best Without the Stress\"\". On August 30, 2017, it was announced that Radke was one of 18 competitors on \"\"\"\", the 35th season of the U.S. version of \"\"Survivor\"\". She was part of the Levu (Heroes) tribe and was the first person voted out of the game. Radke is married to former Stanford University and Olympic swim coach Ross Gerry. Together, they reside in Excelsior, Minnesota, and have two children. Radke works as a marriage and family therapist (MFT), as well as an online professor for Foothill College in Los Altos Hills, California. She"}
+{"id": "306017", "contents": "\"Allies of World War II\"\nUnion also urged military and cooperation between Nationalist China and Communist China during the war. Even though China had been fighting the longest among all the Allied powers, it only officially joined the Allies after the attack on Pearl Harbor, on 7 December 1941. China fought the Japanese Empire before joining the Allies In the Pacific War. Generalissimo Chiang Kai-shek thought Allied victory was assured with the entrance of the United States into the war, and he declared war on Germany and the other Axis nations. However, Allied aid remained low because the Burma Road was closed and the Allies"}
+{"id": "4951746", "contents": "\"Performance medicine\"\nPerformance medicine Performance Medicine is a sub-speciality of clinical and diagnostic medicine that is focussed on the optimisation of emotional, mental, and emotional well-being and performance. It is a new and innovative medical speciality that merges the goals of internal medicine, anti-ageing medicine, functional medicine, sports medicine and preventative health care. Performance medicine differs fundamentally from other areas of medicine by focusing on the healthy non-injured individual that is at low risk of sub-clinical or clinical disease. This population of individuals responds poorly to the current medical paradigm. It uses the principles of evidence-based medicine not to treat disease, but"}
+{"id": "2530998", "contents": "Paskhas\nto provide an air defense capability. In 1950, the Parachute School was also opened to train future Air Force paratroppers at the Andir Air Force Base in Bandung, West Java, as a continuation of the Para School embryo in Maguwo. Its alumni, both officers and airmen, later formed the basis for the \"\"Kompi-kompi Pasukan Gerak Tjepat\"\"-PGT (Quick Reaction Troop Companies) which were formed in February 1952, with Captain Wiriadinata as the commanding officer and concurrently as Commander of Andir Air Force Base in Bandung. By 1958 these consisted of 11 Independent Airborne Companies, 8 Air Base Defense companies and 1"}
+{"id": "8115426", "contents": "\"Nous (Diane Birch album)\"\nstruck by the openness and generosity of the musicians she met in Berlin who were friends and acquaintances of a new friend of hers, drummer Max Weissenfeldt, and she was able to rediscover her creativity. However, she still considers New York to be her home, needing to return even after needing to leave. The album title is derived from a Greek word meaning \"\"mind\"\" or \"\"intellect\"\" but most commonly used as \"\"common sense\"\", \"\"awareness\"\", or \"\"consciousness\"\", and also \"\"us\"\" in French. Birch was drawn by the multifaceted nature of the word. The album deals with issues of love, power, gender,"}
+{"id": "7272002", "contents": "\"Crosscut Saw (song)\"\nversion made it a popular blues standard with blues musicians. It also identifies Al Jackson as the one primarily responsible for bringing the song to Stax and King as well as giving it a Latin (Afro-Cuban) beat. Crosscut Saw (song) \"\"Crosscut Saw\"\", or \"\"Cross Cut Saw Blues\"\" as it was first called, is a dirty blues song \"\"that must have belonged to the general repertoire of the Delta blues\"\". The song was first released in 1941 by Mississippi bluesman Tommy McClennan and has since been interpreted by many blues artists. \"\"Crosscut Saw\"\" became an early R&B chart hit for Albert"}
+{"id": "5976378", "contents": "\"Peabody Magnet High School\"\nIndustrial Training School offering training in home economics and industrial shop to a comprehensive high school offering courses in algebra, geometry, social studies, science, physics, chemistry, art, music, band, Spanish, French, business, auto mechanics, mechanical drawing, woodwork, sheet metal, distributive education, cooperative office education, and speech with an array of extracurricular activities. In August 1972, Mr. Iles retired as principal of Peabody and accepted a position at the Rapides Parish School Board's Central Office. Mr. Samuel McKay, a distinguished chemistry teacher and community leader, succeeded Mr. Iles as the principal of Peabody from 1972 until 1981. Under his leadership, a"}
\ No newline at end of file

dataflow/example/CodePipeline/code_synthesis_input.jsonl

+{"id": "code_001", "raw_code": "def bubble_sort(arr):\n    n = len(arr)\n    for i in range(n):\n        for j in range(0, n-i-1):\n            if arr[j] > arr[j+1]:\n                arr[j], arr[j+1] = arr[j+1], arr[j]"}
+{"id": "code_002", "raw_code": "def factorial(n):\n    if n == 0:\n        return 1\n    else:\n        return n * factorial(n-1)"}
+{"id": "code_003", "raw_code": "def fibonacci(n):\n    if n <= 1:\n        return n\n    return fibonacci(n-1) + fibonacci(n-2)"}
+{"id": "code_004", "raw_code": "def binary_search(arr, target):\n    left, right = 0, len(arr) - 1\n    while left <= right:\n        mid = (left + right) // 2\n        if arr[mid] == target:\n            return mid\n        elif arr[mid] < target:\n            left = mid + 1\n        else:\n            right = mid - 1\n    return -1"}
+{"id": "code_005", "raw_code": "def is_prime(n):\n    if n < 2:\n        return False\n    for i in range(2, int(n**0.5) + 1):\n        if n % i == 0:\n            return False\n    return True"}

dataflow/example/DataflowAgent/agent_test_data.json

+[
+    {"instruction": "Barack Obama was born in Hawaii in 1961."},
+    {"instruction": "2024年巴黎奥运会将在法国巴黎举行。"},
+    {"instruction": "Apple Inc. unveiled the new iPhone at their Cupertino headquarters."},
+    {"instruction": "张伟和李雷于2023年5月在北京天安门广场相遇。"},
+    {"instruction": "The COVID-19 pandemic started in 2019 and changed the world."},
+    {"instruction": "联合国总部位于纽约。"},
+    {"instruction": "Elon Musk announced SpaceX's Mars mission will launch in 2026."},
+    {"instruction": "上海迪士尼乐园自2016年起对外开放。"},
+    {"instruction": "The Nobel Prize in Literature was awarded to Gabriel García Márquez in 1982."},
+    {"instruction": "马云是阿里巴巴集团的创始人。"},
+    {"instruction": "Microsoft was founded by Bill Gates and Paul Allen in 1975."},
+    {"instruction": "乔布斯于2001年推出了第一代iPod。"},
+    {"instruction": "The Eiffel Tower is located in Paris, France."},
+    {"instruction": "新冠疫苗于2020年底在全球范围内开始接种。"},
+    {"instruction": "Angela Merkel served as the Chancellor of Germany until 2021."},
+    {"instruction": "北京大学是中国著名的高等学府。"},
+    {"instruction": "Tesla delivered over one million cars in 2022."},
+    {"instruction": "2022年卡塔尔世界杯吸引了全球大量球迷。"},
+    {"instruction": "Mark Zuckerberg is the CEO of Meta Platforms."},
+    {"instruction": "阿里巴巴集团于1999年在杭州成立。"},
+    {"instruction": "The Grand Canyon is a famous tourist attraction in the United States."},
+    {"instruction": "刘亦菲主演的电影《花木兰》于2020年上映。"},
+    {"instruction": "NASA plans to return to the Moon by 2025."},
+    {"instruction": "小米公司于2010年由雷军创立。"},
+    {"instruction": "The Great Wall of China is one of the Seven Wonders of the World."},
+    {"instruction": "2021年东京奥运会因疫情推迟一年举行。"},
+    {"instruction": "Jeff Bezos stepped down as Amazon's CEO in July 2021."},
+    {"instruction": "清华大学位于北京市海淀区。"},
+    {"instruction": "Uber launched its ride-sharing service in San Francisco in 2010."},
+    {"instruction": "林书豪在2012年掀起了“林疯狂”热潮。"},
+    {"instruction": "In the early morning of July 20, 1969, Neil Armstrong became the first human to set foot on the lunar surface, uttering the famous words, 'That's one small step for man, one giant leap for mankind.' This historic event was broadcast live to millions around the world."},
+    {"instruction": "据世界卫生组织统计，全球每年有数百万人因空气污染而提前死亡。各国政府正积极采取措施，推动新能源和清洁技术的发展，以减缓环境恶化和气候变化。"},
+    {"instruction": "During the Renaissance period, artists such as Leonardo da Vinci and Michelangelo revolutionized European art with their innovative techniques and profound understanding of human anatomy, paving the way for future generations."},
+    {"instruction": "2023年，中国高铁总里程突破4万公里，成为全球高铁网络最长的国家。高铁的发展不仅极大便利了人们的出行，也带动了沿线城市的经济增长。"},
+    {"instruction": "After years of research, scientists at CERN finally announced the discovery of the Higgs boson particle in 2012, confirming a major prediction of the Standard Model of particle physics and opening new avenues for scientific exploration."},
+    {"instruction": "玛丽·居里因其在放射性领域的卓越贡献，成为历史上首位两次获得诺贝尔奖的科学家，她的研究成果对医学和物理学产生了深远影响。"},
+    {"instruction": "The Amazon rainforest, often referred to as the 'lungs of the Earth,' is home to an astonishing variety of plant and animal species. However, deforestation poses a serious threat to this vital ecosystem."},
+    {"instruction": "2022年，北京冬奥会在全球瞩目下成功举办，来自90多个国家和地区的运动员在冰雪赛场上奋勇拼搏，展现了奥林匹克精神。"},
+    {"instruction": "On September 5, 1977, NASA launched the Voyager 1 spacecraft, which has since become the most distant human-made object in space, sending back invaluable data from the outer reaches of our solar system."},
+    {"instruction": "随着人工智能和大数据技术的发展，智能助手正在逐步进入人们的日常生活，从语音识别到智能家居，极大地提升了生活便利性。"},
+    {"instruction": "In 1994, Nelson Mandela was elected as South Africa’s first Black president, marking the end of decades of apartheid and setting an example of reconciliation and national unity for the world."},
+    {"instruction": "2010年上海世博会以“城市，让生活更美好”为主题，吸引了来自世界各地的游客和参展国家，展现了全球城市化进程的最新成果。"},
+    {"instruction": "The Internet, which started as a research project in the 1960s, has evolved into a global network connecting billions of people, revolutionizing communication, commerce, and information sharing."},
+    {"instruction": "2018年，嫦娥四号成功登陆月球背面，成为人类历史上首个在月球远端实现软着陆的探测器，为月球科学研究提供了宝贵数据。"},
+    {"instruction": "The story of Romeo and Juliet, written by William Shakespeare, has been adapted into countless plays, films, and ballets, resonating with audiences for its portrayal of love and tragedy."},
+    {"instruction": "近年来，随着健康意识的提升，越来越多的人选择通过运动和科学饮食来改善生活方式，健身房和健康食品市场呈现出快速增长的趋势。"},
+    {"instruction": "In 2008, the global financial crisis triggered a severe economic downturn, leading to widespread unemployment, bankruptcies, and significant reforms in the banking and financial sectors."},
+    {"instruction": "2020年新冠疫情爆发后，全球多国实施了严格的旅行限制和居家隔离措施，推动了远程办公和在线教育的快速普及。"},
+    {"instruction": "Mount Everest, the world’s highest peak, attracts climbers from around the globe who brave extreme conditions in pursuit of the summit, often facing challenges such as avalanches and altitude sickness."},
+    {"instruction": "大熊猫是中国的国宝，因其独特的黑白外貌和温顺的性格深受人们喜爱。近年来，保护大熊猫及其栖息地已成为全球共同关注的话题。"},
+    {"instruction": "In the realm of computer science, the development of quantum computing promises to solve complex problems that are currently intractable for classical computers, potentially transforming industries ranging from cryptography to pharmaceuticals."},
+    {"instruction": "2021年，东京奥运会在推迟一年后如期举行，尽管受到疫情影响，运动员们依然展现了顽强的拼搏精神和卓越的竞技水平。"},
+    {"instruction": "The Sahara Desert, spanning across North Africa, is the largest hot desert in the world. Its harsh environment has shaped the cultures and lifestyles of the people living on its fringes."},
+    {"instruction": "随着新能源汽车技术的进步，电动汽车在全球范围内逐渐普及，许多国家出台了鼓励政策，推动绿色出行和环境保护。"},
+    {"instruction": "In the world of literature, magical realism is a genre that combines realistic narrative with surreal elements, as exemplified by the works of Gabriel García Márquez and Haruki Murakami."},
+    {"instruction": "2019年，5G商用网络在中国、美国、韩国等国家正式上线，为移动互联网和物联网的发展带来了全新的机遇和挑战。"},
+    {"instruction": "The Great Barrier Reef, located off the coast of Australia, is the largest coral reef system on Earth, but it faces significant threats from climate change and pollution."},
+    {"instruction": "近年来，短视频平台的兴起极大地改变了人们的娱乐和社交方式，推动了内容创作和数字营销的新模式。"},
+    {"instruction": "In 1963, Martin Luther King Jr. delivered his iconic 'I Have a Dream' speech during the March on Washington, becoming a symbol of the civil rights movement in the United States."},
+    {"instruction": "2022年，全球气候变暖问题持续加剧，各地极端天气频发，科学家呼吁各国加强合作，共同应对环境挑战。"},
+    {"instruction": "The International Space Station, a collaborative effort between multiple countries, serves as a microgravity laboratory for scientific research and technological innovation."},
+    {"instruction": "在中国传统文化中，春节是最重要的节日之一，人们通过贴春联、放鞭炮、团圆饭等方式庆祝新年，祈求幸福安康。"},
+    {"instruction": "Advancements in medical technology, such as gene editing and personalized medicine, hold great promise for the treatment of previously incurable diseases."},
+    {"instruction": "2023年，全球芯片短缺问题对汽车、电子和通信等行业产生了重大影响，企业纷纷调整供应链策略以应对危机。"},
+    {"instruction": "The Louvre Museum in Paris is home to some of the world’s most famous artworks, including the Mona Lisa and the Venus de Milo, attracting millions of visitors each year."},
+    {"instruction": "近年来，随着环保理念的普及，越来越多的企业开始采用可再生能源和绿色生产工艺，推动可持续发展。"},
+    {"instruction": "In 2017, astronomers detected gravitational waves from the collision of two neutron stars, providing new insights into the origins of heavy elements in the universe."},
+    {"instruction": "2022年中国航天员乘坐神舟十四号飞船成功进入中国空间站，进行一系列科学实验和空间建设任务，标志着中国载人航天进入新阶段。"},
+    {"instruction": "The ancient city of Rome, with its rich history and architectural marvels such as the Colosseum and the Pantheon, continues to fascinate tourists and historians alike."},
+    {"instruction": "近年来，心理健康问题受到社会广泛关注，越来越多的人开始重视心理疏导和压力管理，心理咨询服务需求不断增长。"},
+    {"instruction": "Climate activists around the globe have organized rallies and campaigns to raise awareness about environmental issues, urging governments and corporations to take meaningful action."}
+]
\ No newline at end of file

dataflow/example/FuncCallPipeline/chat_data.jsonl

+{"system": "SYSTEM: You are a helpful assistant with access to the following functions. Use them if required -\n{\n    \"name\": \"get_exchange_rate\",\n    \"description\": \"Get the exchange rate between two currencies\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"base_currency\": {\n                \"type\": \"string\",\n                \"description\": \"The currency to convert from\"\n            },\n            \"target_currency\": {\n                \"type\": \"string\",\n                \"description\": \"The currency to convert to\"\n            }\n        },\n        \"required\": [\n            \"base_currency\",\n            \"target_currency\"\n        ]\n    }\n}\n", "chat": "USER: Can you book a flight for me from New York to London?\n\n\nASSISTANT: I'm sorry, but I don't have the capability to book flights. My current function allows me to get the exchange rate between two currencies. If you need help with that, feel free to ask! <|endoftext|>\n\n\n"}
+{"system": "SYSTEM: You are a helpful assistant with access to the following functions. Use them if required -\n{\n    \"name\": \"get_news_headlines\",\n    \"description\": \"Get the latest news headlines\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"country\": {\n                \"type\": \"string\",\n                \"description\": \"The country for which to fetch news\"\n            }\n        },\n        \"required\": [\n            \"country\"\n        ]\n    }\n}\n", "chat": "USER: Can you tell me the latest news headlines for the United States?\n\n\nASSISTANT: <functioncall> {\"name\": \"get_news_headlines\", \"arguments\": '{\"country\": \"United States\"}'} <|endoftext|>\n\n\nFUNCTION RESPONSE: {\"headlines\": [\"Biden announces new vaccine mandates\", \"Hurricane Ida devastates Louisiana\", \"Apple unveils new iPhone\", \"NASA's Perseverance rover collects first Mars rock sample\"]}\n\n\nASSISTANT: Here are the latest news headlines for the United States:\n1. Biden announces new vaccine mandates\n2. Hurricane Ida devastates Louisiana\n3. Apple unveils new iPhone\n4. NASA's Perseverance rover collects first Mars rock sample <|endoftext|>\n\n\nUSER: That's interesting. What about the news in France?\n\n\nASSISTANT: <functioncall> {\"name\": \"get_news_headlines\", \"arguments\": '{\"country\": \"France\"}'} <|endoftext|>\n\n\nFUNCTION RESPONSE: {\"headlines\": [\"France recalls ambassadors to US and Australia\", \"French election: Macron's party braces for tough fight\", \"Louvre Museum to undergo major overhaul\", \"France to offer free birth control to all women under 25\"]}\n\n\nASSISTANT: Here are the latest news headlines for France:\n1. France recalls ambassadors to US and Australia\n2. French election: Macron's party braces for tough fight\n3. Louvre Museum to undergo major overhaul\n4. France to offer free birth control to all women under 25 <|endoftext|>\n\n\n"}
+{"system": "SYSTEM: You are a helpful assistant with access to the following functions. Use them if required -\n{\n    \"name\": \"generate_password\",\n    \"description\": \"Generate a random password\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"length\": {\n                \"type\": \"integer\",\n                \"description\": \"The length of the password\"\n            },\n            \"include_symbols\": {\n                \"type\": \"boolean\",\n                \"description\": \"Whether to include symbols in the password\"\n            }\n        },\n        \"required\": [\n            \"length\"\n        ]\n    }\n}\n\n{\n    \"name\": \"create_task\",\n    \"description\": \"Create a new task in a task management system\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"title\": {\n                \"type\": \"string\",\n                \"description\": \"The title of the task\"\n            },\n            \"due_date\": {\n                \"type\": \"string\",\n                \"format\": \"date\",\n                \"description\": \"The due date of the task\"\n            },\n            \"priority\": {\n                \"type\": \"string\",\n                \"enum\": [\n                    \"low\",\n                    \"medium\",\n                    \"high\"\n                ],\n                \"description\": \"The priority of the task\"\n            }\n        },\n        \"required\": [\n            \"title\",\n            \"due_date\",\n            \"priority\"\n        ]\n    }\n}\n\n", "chat": "USER: I need a new password. Can you generate one for me?\n\n\nASSISTANT: Of course. How long would you like your password to be? And would you like it to include symbols? <|endoftext|>\n\n\nUSER: I would like it to be 12 characters long and yes, please include symbols.\n\n\nASSISTANT: <functioncall> {\"name\": \"generate_password\", \"arguments\": '{\"length\": 12, \"include_symbols\": true}'} <|endoftext|>\n\n\nFUNCTION RESPONSE: {\"password\": \"4&7j#9@1Q6*\"}\n\n\nASSISTANT: Here is your new password: 4&7j#9@1Q6*. Please make sure to save it in a secure location. <|endoftext|>\n\n\n"}
+{"system": "SYSTEM: You are a helpful assistant with access to the following functions. Use them if required -\n{\n    \"name\": \"generate_password\",\n    \"description\": \"Generate a random password with specified criteria\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"length\": {\n                \"type\": \"integer\",\n                \"description\": \"The length of the password\"\n            },\n            \"include_numbers\": {\n                \"type\": \"boolean\",\n                \"description\": \"Include numbers in the password\"\n            },\n            \"include_special_characters\": {\n                \"type\": \"boolean\",\n                \"description\": \"Include special characters in the password\"\n            }\n        },\n        \"required\": [\n            \"length\"\n        ]\n    }\n}\n", "chat": "USER: I need a new password. Can you generate one for me?\n\n\nASSISTANT: Of course! How long would you like your password to be? And do you want it to include numbers and special characters? <|endoftext|>\n\n\nUSER: I want it to be 12 characters long and yes, it should include both numbers and special characters.\n\n\nASSISTANT: <functioncall> {\"name\": \"generate_password\", \"arguments\": '{\"length\": 12, \"include_numbers\": true, \"include_special_characters\": true}'} <|endoftext|>\n\n\nFUNCTION RESPONSE: {\"password\": \"4#7gB6&9L1!0\"}\n\n\nASSISTANT: Here is your new password: 4#7gB6&9L1!0. Please make sure to save it in a secure place. <|endoftext|>\n\n\n"}
+{"system": "SYSTEM: You are a helpful assistant with access to the following functions. Use them if required -\n{\n    \"name\": \"calculate_median\",\n    \"description\": \"Calculate the median of a list of numbers\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"numbers\": {\n                \"type\": \"array\",\n                \"items\": {\n                    \"type\": \"number\"\n                },\n                \"description\": \"A list of numbers\"\n            }\n        },\n        \"required\": [\n            \"numbers\"\n        ]\n    }\n}\n", "chat": "USER: Hi, I have a list of numbers and I need to find the median. Can you help me with that?\n\n\nASSISTANT: Of course, I can help you with that. Please provide me with the list of numbers. <|endoftext|>\n\n\nUSER: The numbers are 5, 2, 9, 1, 7, 4, 6, 3, 8.\n\n\nASSISTANT: <functioncall> {\"name\": \"calculate_median\", \"arguments\": '{\"numbers\": [5, 2, 9, 1, 7, 4, 6, 3, 8]}'} <|endoftext|>\n\n\nFUNCTION RESPONSE: {\"median\": 5}\n\n\nASSISTANT: The median of the provided list of numbers is 5. <|endoftext|>\n\n\n"}

dataflow/example/GeneralTextPipeline/abbreviation.jsonl

+{"raw_content": "In recent years, the field of natural language processing has witnessed significant progress driven by the advent of large language models. These models, trained on massive corpora, have demonstrated remarkable capabilities across various tasks, including text generation, summarization, and question answering. However, despite their success, concerns remain about their data efficiency, generalization to low-resource languages, and robustness under distributional shifts."}
+{"raw_content": "This study presents a comprehensive evaluation of domain-adaptive pretraining techniques for transformer-based models. By integrating task-relevant corpora into the pretraining stage, we observe consistent improvements across multiple benchmarks. Our findings highlight the importance of targeted data selection and emphasize that not all additional data contributes equally to downstream performance."}
+{"raw_content": "We propose an efficient multi-stage training pipeline that combines synthetic data generation, automatic filtering, and human-in-the-loop refinement. The system is designed to minimize annotation costs while maintaining high quality and diversity in the resulting dataset. Experiments show that our approach achieves comparable performance to fully human-labeled datasets in both classification and generation tasks."}
+{"raw_content": "The rise of multimodal AI systems has enabled models to process and reason over diverse input formats such as text, images, and audio. In this work, we introduce a unified data framework that standardizes input preprocessing, alignment, and batching for multimodal tasks, significantly simplifying the training and evaluation of vision-language models."}
+{"raw_content": "To address the limitations of one-pass data usage in current large model training paradigms, we explore strategies for selective data reuse. By identifying high-value samples through gradient-based scoring, we propose a curriculum that dynamically reintroduces informative examples during training. This method improves convergence speed and yields better generalization on challenging benchmarks."}
+{"raw_content": "We conduct a rigorous analysis of prompt-based learning in cross-lingual settings. Our results show that prompt templates optimized for English often fail to transfer to other languages due to differences in syntax and pragmatics. We thus introduce a language-sensitive prompt optimization framework that outperforms traditional zero-shot transfer methods by a wide margin."}
+{"raw_content": "Our research investigates the scaling behavior of multimodal large language models when jointly increasing visual encoder depth and language model capacity. Through extensive experiments across datasets like VQAv2, TextVQA, and ScienceQA, we reveal trade-offs between modality balance and cross-attention saturation, offering practical guidelines for future model design."}
+{"raw_content": "This paper introduces a reference-free evaluation metric for long-form summarization tasks. Unlike traditional ROUGE or BLEU scores, our method leverages semantic similarity via pretrained sentence encoders and evaluates coherence, coverage, and conciseness. Human evaluations show a stronger correlation with our metric compared to existing automatic methods."}
+{"raw_content": "We explore the use of reinforcement learning to fine-tune dialogue models for empathy. Using a composite reward function based on emotional alignment, relevance, and fluency, our system learns to generate more supportive and human-like responses. The results from both automatic metrics and human evaluations demonstrate substantial gains in perceived empathy and helpfulness."}
+{"raw_content": "Our work presents a novel video question answering benchmark with temporally grounded annotations. We design a new dataset with fine-grained reasoning chains and dense frame references to test model understanding of temporal dependencies. Baseline models struggle with multi-hop temporal reasoning, highlighting the need for more temporally aware architectures."}

dataflow/example/GeneralTextPipeline/filtering.json

+[
+  {
+    "score": 0.9,
+    "status": "valid",
+    "comment": "Good and excellent effort done",
+    "remark": null
+  },
+  {
+    "score": 1.0,
+    "status": "pending",
+    "comment": "Needs review",
+    "remark": "note"
+  },
+  {
+    "score": 0.75,
+    "status": "reviewed",
+    "comment": "excellent review completed",
+    "remark": "confirmed"
+  },
+  {
+    "score": 1.1,
+    "status": "rejected",
+    "comment": "all done",
+    "remark": null
+  },
+  {
+    "score": 0.7,
+    "status": "unknown",
+    "comment": null,
+    "remark": "okay"
+  }
+]

dataflow/example/GeneralTextPipeline/gen_input.jsonl

+{"input_key": "The quick brown fox jumps over the lazy dog.", "reference_key": "A fast brown fox leaps over a lazy dog."}
+{"input_key": "She sells seashells by the seashore.", "reference_key": "She is selling shells by the beach."}
+{"input_key": "To be or not to be, that is the question.", "reference_key": "The question is whether to be or not."}
+{"input_key": "All that glitters is not gold.", "reference_key": "Not everything that shines is gold."}
+{"input_key": "A picture is worth a thousand words.", "reference_key": "A single image can convey so much meaning."}
+{"input_key": "Actions speak louder than words.", "reference_key": "What people do is more important than what they say."}
+{"input_key": "Beauty is in the eye of the beholder.", "reference_key": "What one person finds beautiful may differ from others."}
+{"input_key": "The early bird catches the worm.", "reference_key": "People who wake up early often achieve more."}
+{"input_key": "Two heads are better than one.", "reference_key": "Collaborating is more effective than working alone."}
+{"input_key": "A journey of a thousand miles begins with a single step.", "reference_key": "Big achievements start with small actions."}