适配后端vllm

dataflow/example/core_text_data/double_column_input.json

+[
+    {
+        "roll": "pig",
+        "term": "eat"
+    },
+    {
+        "roll": "tiger",
+        "term": "chase"
+    },
+    {
+        "roll": "people",
+        "term": "drink"
+    },
+    {
+        "roll": "bird",
+        "term": "dance"
+    }
+]
\ No newline at end of file

dataflow/example/core_text_data/eval_data.json

+[
+  {
+    "conversations": [
+      {"from": "human", "value": "Hi, can you explain what machine learning is?"},
+      {"from": "gpt", "value": "Sure! Machine learning is a field of artificial intelligence where systems learn patterns from data and improve their performance without being explicitly programmed."},
+      {"from": "human", "value": "Can you give me a simple example?"},
+      {"from": "gpt", "value": "Of course. A classic example is spam email detection. The system learns from many examples of spam and non-spam emails, then predicts whether new emails are spam."}
+    ]
+  },
+  {
+    "conversations": [
+      {"from": "human", "value": "What is the capital of France?"},
+      {"from": "gpt", "value": "The capital of France is Paris."},
+      {"from": "human", "value": "And what about Germany?"},
+      {"from": "gpt", "value": "Germany’s capital is Berlin."}
+    ]
+  },
+  {
+    "conversations": [
+      {"from": "human", "value": "帮我写一个Python函数，计算两个数的最大公约数"},
+      {"from": "gpt", "value": "可以使用欧几里得算法：\n```python\ndef gcd(a, b):\n    while b:\n        a, b = b, a % b\n    return a\n```"},
+      {"from": "human", "value": "能举个例子吗？"},
+      {"from": "gpt", "value": "当然，例如 `gcd(48, 18)` 会返回 `6`。"}
+    ]
+  },
+  {
+    "conversations": [
+      {"from": "human", "value": "What are the benefits of regular exercise?"},
+      {"from": "gpt", "value": "Regular exercise improves cardiovascular health, strengthens muscles, boosts mood, reduces stress, and can help maintain a healthy weight."},
+      {"from": "human", "value": "How many times per week is recommended?"},
+      {"from": "gpt", "value": "Most guidelines suggest at least 150 minutes of moderate-intensity exercise per week, which can be split into about 30 minutes on 5 days."}
+    ]
+  },
+  {
+    "conversations": [
+      {"from": "human", "value": "请解释一下量子计算机和经典计算机的区别"},
+      {"from": "gpt", "value": "经典计算机使用比特(bit)，每个比特只能是0或1；而量子计算机使用量子比特(qubit)，它可以同时处于0和1的叠加态。"},
+      {"from": "human", "value": "那它为什么更强大？"},
+      {"from": "gpt", "value": "因为量子叠加和量子纠缠，使得量子计算机能并行处理大量计算问题，在某些任务上比经典计算机快得多。"}
+    ]
+  }
+]

dataflow/example/core_text_data/pipeline_small_chunk.json

+[
+    {
+        "text": "What is anatomy? Anatomy includes those structures that can be seen grossly (without the aid of magnification) and microscopically (with the aid of magnification). Typically, when used by itself, the term anatomy tends to mean gross or macroscopic anatomy—that is, the study of structures that can be seen without using a microscopic. Microscopic anatomy, also called histology, is the study of cells and tissues using a microscope. Anatomy forms the basis for the practice of medicine. Anatomy leads the physician toward an understanding of a patient’s disease, whether he or she is carrying out a physical examination or using the most advanced imaging techniques. Anatomy is also important for dentists, chiropractors, physical therapists, and all others involved in any aspect of patient treatment that begins with an analysis of clinical signs. The ability to interpret a clinical observation correctly is therefore the endpoint of a sound anatomical understanding. Observation and visualization are the primary techniques a student should use to learn anatomy. Anatomy is much more than just memorization of lists of names. Although the language of anatomy is important, the network of information needed to visualize the position of physical structures in a patient goes far beyond simple memorization. Knowing the names of the various branches of the external carotid artery is not the same as being able to visualize the course of the lingual artery from its origin in the neck to its termination in the tongue. Similarly, understanding the organization of the soft palate, how it is related to the oral and nasal cavities, and how it moves during swallowing is very different from being able to recite the names of its individual muscles and nerves. An understanding of anatomy requires an understanding of the context in which the terminology can be remembered. How can gross anatomy be studied? The term anatomy is derived from the Greek word temnein, meaning “to cut.” Clearly, therefore, the study of anatomy is linked, at its root, to dissection, although dissection of cadavers by students is now augmented, or even in some cases replaced, by viewing prosected (previously dissected) material and plastic models, or using computer teaching modules and other learning aids. Anatomy can be studied following either a regional or a systemic approach. With a regional approach, each region of the body is studied separately and all aspects of that region are studied at the same time. For example, if the thorax is to be studied, all of its structures are examined. This includes the vasculature, the nerves, the bones, the muscles, and all other structures and organs located in the region of the body defined as the thorax. After studying this region, the other regions of the body (i.e., the abdomen, pelvis, lower limb, upper limb, back, head, and neck) are studied in a similar fashion. In contrast, in a systemic approach, each system of the body is studied and followed throughout the entire body. For example, a study of the cardiovascular system looks at the heart and all of the blood vessels in the body. When this is completed, the nervous system (brain, spinal cord, and all the nerves) might be examined in detail. This approach continues for the whole body until every system, including the nervous, skeletal, muscular, gastrointestinal, respiratory, lymphatic, and reproductive systems, has been studied. Each of these approaches has benefits and deficiencies. The regional approach works very well if the anatomy course involves cadaver dissection but falls short when it comes to understanding the continuity of an entire system throughout the body. Similarly, the systemic approach fosters an understanding of an entire system throughout the body, but it is very difficult to coordinate this directly with a cadaver dissection or to acquire sufficient detail. The anatomical position The anatomical position is the standard reference position of the body used to describe the location of structures (Fig. 1.1). The body is in the anatomical position when standing upright with feet together, hands by the side and face looking forward. The mouth is closed and the facial expression is neutral. The rim of bone under the eyes is in the same horizontal plane as the top of the opening to the ear, and the eyes are open and focused on something in the distance. The palms of the hands face forward with the fingers straight and together and with the pad of the thumb turned 90° to the pads of the fingers. The toes point forward. Three major groups of planes pass through the body in the anatomical position (Fig. 1.1). Coronal planes are oriented vertically and divide the body into anterior and posterior parts. Sagittal planes also are oriented vertically but are at right angles to the coronal planes and divide the body into right and left parts. The plane that passes through the center of the body dividing it into equal right and left halves is termed the median sagittal plane. Transverse, horizontal, or axial planes divide the body into superior and inferior parts. Terms to describe location Anterior (ventral) and posterior (dorsal), medial and lateral, superior and inferior Three major pairs of terms are used to describe the location of structures relative to the body as a whole or to other structures (Fig. 1.1). Anterior (or ventral) and posterior (or dorsal) describe the position of structures relative to the “front” and “back” of the body. For example, the nose is an anterior (ventral) structure, whereas the vertebral column is a posterior (dorsal) structure. Also, the nose is anterior to the ears and the vertebral column is posterior to the sternum. Medial and lateral describe the position of structures relative to the median sagittal plane and the sides of the body. For example, the thumb is lateral to the little finger. The nose is in the median sagittal plane and is medial to the eyes, which are in turn medial to the external ears. Superior and inferior describe structures in reference to the vertical axis of the body. For example, the head is superior to the shoulders and the knee joint is inferior to the hip joint. Proximal and distal, cranial and caudal, Other terms used to describe positions include proximal and distal, cranial and caudal, and rostral. Proximal and distal are used with reference to being closer to or farther from a structure’s origin, particularly in the limbs. For example, the hand is distal to the elbow joint. The glenohumeral joint is proximal to the elbow joint. These terms are also used to describe the relative positions of branches along the course of linear structures, such as airways, vessels, and nerves. For example, distal branches occur farther away toward the ends of the system, whereas proximal branches occur closer to and toward the origin of the system. Cranial (toward the head) and caudal (toward the tail) are sometimes used instead of superior and inferior, respectively. Rostral is used, particularly in the head, to describe the position of a structure with reference to the nose. For example, the forebrain is rostral to the hindbrain. Two other terms used to describe the position of structures in the body are superficial and deep. These terms are used to describe the relative positions of two structures with respect to the surface of the body. For example, the sternum is superficial to the heart, and the stomach is deep to the abdominal wall. Superficial and deep can also be used in a more absolute fashion to define two major regions of the body. The superficial region of the body is external to the outer layer of deep fascia. Deep structures are enclosed by this layer. Structures in the superficial region of the body include the skin, superficial fascia, and mammary glands. Deep structures include most skeletal muscles and viscera. Superficial wounds are external to the outer layer of deep fascia, whereas deep wounds penetrate through it. In 1895 Wilhelm Roentgen used the X-rays from a cathode ray tube to expose a photographic plate and produce the first radiographic exposure of his wife’s hand. Over the past 35 years there has been a revolution in body imaging, which has been paralleled by developments in computer technology. X-rays are photons (a type of electromagnetic radiation) and are generated from a complex X-ray tube, which is a type of cathode ray tube (Fig. 1.2). The X-rays are then collimated (i.e., directed through lead-lined shutters to stop them from fanning out) to the appropriate area of the body. As the X-rays pass through the body they are attenuated (reduced in energy) by the tissues. Those X-rays that pass through the tissues interact with the photographic film. In the body: air attenuates X-rays a little; fat attenuates X-rays more than air but less than bone attenuates X-rays the most. These differences in attenuation result in differences in the level of exposure of the film. When the photographic film is developed, bone appears white on the film because this region of the film has been exposed to the least amount of X-rays. Air appears dark on the film because these regions were exposed to the greatest number of X-rays. Modifications to this X-ray technique allow a continuous stream of X-rays to be produced from the X-ray tube and collected on an input screen to allow real-time visualization of moving anatomical structures, barium studies, angiography, and fluoroscopy (Fig. 1.3). To demonstrate specific structures, such as bowel loops or arteries, it may be necessary to fill these structures with a substance that attenuates X-rays more than bowel loops or arteries do normally. It is, however, extremely important that these substances are nontoxic. Barium sulfate, an insoluble salt, is a nontoxic, relatively high-density agent that is extremely useful in the examination of the gastrointestinal tract. When a barium sulfate suspension is ingested it attenuates"
+    },
+    {
+        "text": "tissues interact with the photographic film. In the body: air attenuates X-rays a little; fat attenuates X-rays more than air but less than bone attenuates X-rays the most. These differences in attenuation result in differences in the level of exposure of the film. When the photographic film is developed, bone appears white on the film because this region of the film has been exposed to the least amount of X-rays. Air appears dark on the film because these regions were exposed to the greatest number of X-rays. Modifications to this X-ray technique allow a continuous stream of X-rays to be produced from the X-ray tube and collected on an input screen to allow real-time visualization of moving anatomical structures, barium studies, angiography, and fluoroscopy (Fig. 1.3). To demonstrate specific structures, such as bowel loops or arteries, it may be necessary to fill these structures with a substance that attenuates X-rays more than bowel loops or arteries do normally. It is, however, extremely important that these substances are nontoxic. Barium sulfate, an insoluble salt, is a nontoxic, relatively high-density agent that is extremely useful in the examination of the gastrointestinal tract. When a barium sulfate suspension is ingested it attenuates X-rays and can therefore be used to demonstrate the bowel lumen (Fig. 1.4). It is common to add air to the barium sulfate suspension, by either ingesting “fizzy” granules or directly instilling air into the body cavity, as in a barium enema. This is known as a double-contrast (air/barium) study. For some patients it is necessary to inject contrast agents directly into arteries or veins. In this case, iodine-based molecules are suitable contrast agents. Iodine is chosen because it has a relatively high atomic mass and so markedly attenuates X-rays, but also, importantly, it is naturally excreted via the urinary system. Intra-arterial and intravenous contrast agents are extremely safe and are well tolerated by most patients. Rarely, some patients have an anaphylactic reaction to intra-arterial or intravenous injections, so the necessary precautions must be taken. Intra-arterial and intravenous contrast agents not only help in visualizing the arteries and veins but because they are excreted by the urinary system, can also be used to visualize the kidneys, ureter, and bladder in a process known as intravenous urography. During angiography it is often difficult to appreciate the contrast agent in the vessels through the overlying bony structures. To circumvent this, the technique of subtraction angiography has been developed. Simply, one or two images are obtained before the injection of contrast media. These images are inverted (such that a negative is created from the positive image). After injection of the contrast media into the vessels, a further series of images are obtained, demonstrating the passage of the contrast through the arteries into the veins and around the circulation. By adding the “negative precontrast image” to the positive postcontrast images, the bones and soft tissues are subtracted to produce a solitary image of contrast only. Before the advent of digital imaging this was a challenge, but now the use of computers has made this technique relatively straightforward and instantaneous (Fig. 1.5). Ultrasonography of the body is widely used for all aspects of medicine. Ultrasound is a very high frequency sound wave (not electromagnetic radiation) generated by piezoelectric materials, such that a series of sound waves is produced. Importantly, the piezoelectric material can also receive the sound waves that bounce back from the internal organs. The sound waves are then interpreted by a powerful computer, and a real-time image is produced on the display panel. Developments in ultrasound technology, including the size of the probes and the frequency range, mean that a broad range of areas can now be scanned. Traditionally ultrasound is used for assessing the abdomen (Fig. 1.6) and the fetus in pregnant women. Ultrasound is also widely used to assess the eyes, neck, soft tissues, and peripheral musculoskeletal system. Probes have been placed on endoscopes, and endoluminal ultrasound of the esophagus, stomach, and duodenum is now routine. Endocavity ultrasound is carried out most commonly to assess the genital tract in women using a transvaginal or transrectal route. In men, transrectal ultrasound is the imaging method of choice to assess the prostate in those with suspected prostate hypertrophy or malignancy. Doppler ultrasound enables determination of flow, its direction, and its velocity within a vessel using simple ultrasound techniques. Sound waves bounce off moving structures and are returned. The degree of frequency shift determines whether the object is moving away from or toward the probe and the speed at which it is traveling. Precise measurements of blood flow and blood velocity can therefore be obtained, which in turn can indicate sites of blockage in blood vessels. Computed tomography (CT) was invented in the 1970s by Sir Godfrey Hounsfield, who was awarded the Nobel Prize in Medicine in 1979. Since this inspired invention there have been many generations of CT scanners. A CT scanner obtains a series of images of the body (slices) in the axial plane. The patient lies on a bed, an X-ray tube passes around the body (Fig. 1.7), and a series of images are obtained. A computer carries out a complex mathematical transformation on the multitude of images to produce the final image (Fig. 1.8). Nuclear magnetic resonance imaging was first described in 1946 and used to determine the structure of complex molecules. The process of magnetic resonance imaging (MRI) is dependent on the free protons in the hydrogen nuclei in molecules of water (H2O). Because water is present in almost all biological tissues, the hydrogen proton is ideal. The protons within a patient’s hydrogen nuclei can be regarded as small bar magnets, which are randomly oriented in space. The patient is placed in a strong magnetic field, which aligns the bar magnets. When a pulse of radio waves is passed through the patient the magnets are deflected, and as they return to their aligned position they emit small radio pulses. The strength and frequency of the emitted pulses and the time it takes for the protons to return to their pre-excited state produce a signal. These signals are analyzed by a powerful computer, and an image is created (Fig. 1.9). By altering the sequence of pulses to which the protons are subjected, different properties of the protons can be assessed. These properties are referred to as the “weighting” of the scan. By altering the pulse sequence and the scanning parameters, T1-weighted images (Fig. 1.10A) and T2-weighted images (Fig. 1.10B) can be obtained. These two types of imaging sequences provide differences in image contrast, which accentuate and optimize different tissue characteristics. From the clinical point of view: Most T1-weighted images show dark fluid and bright fat—for example, within the brain the cerebrospinal fluid (CSF) is dark. T2-weighted images demonstrate a bright signal from fluid and an intermediate signal from fat—for example, in the brain the CSF appears white. MRI can also be used to assess flow within vessels and to produce complex angiograms of the peripheral and cerebral circulation. Diffusion-weighted imaging provides information on the degree of Brownian motion of water molecules in various tissues. There is relatively free diffusion in extracellular spaces and more restricted diffusion in intracellular spaces. In tumors and infarcted tissue, there is an increase in intracellular fluid water molecules compared with the extracellular fluid environment resulting in overall increased restricted diffusion, and therefore identification of abnormal from normal tissue. Nuclear medicine involves imaging using gamma rays, which are another type of electromagnetic radiation. The important difference between gamma rays and X-rays is that gamma rays are produced from within the nucleus of an atom when an unstable nucleus decays, whereas X-rays are produced by bombarding an atom with electrons. For an area to be visualized, the patient must receive a gamma ray emitter, which must have a number of properties to be useful, including: a reasonable half-life (e.g., 6 to 24 hours), an easily measurable gamma ray, and energy deposition in as low a dose as possible in the patient’s tissues. The most commonly used radionuclide (radioisotope) is technetium-99m. This may be injected as a technetium salt or combined with other complex molecules. For example, by combining technetium-99m with methylene diphosphonate (MDP), a radiopharmaceutical is produced. When injected into the body this radiopharmaceutical specifically binds to bone, allowing assessment of the skeleton. Similarly, combining technetium-99m with other compounds permits assessment of other parts of the body, for example the urinary tract and cerebral blood flow. Depending on how the radiopharmaceutical is absorbed, distributed, metabolized, and excreted by the body after injection, images are obtained using a gamma camera (Fig. 1.11). Positron emission tomography (PET) is an imaging modality for detecting positron-emitting radionuclides. A positron is an anti-electron, which is a positively charged particle of antimatter. Positrons are emitted from the decay of proton-rich radionuclides. Most of these radionuclides are made in a cyclotron and have extremely short half-lives. The most commonly used PET radionuclide is fluorodeoxyglucose (FDG) labeled with fluorine-18 (a positron emitter). Tissues that are actively metabolizing glucose take up this compound, and the resulting localized high concentration of"
+    },
+    {
+        "text": "ium-99m with methylene diphosphonate (MDP), a radiopharmaceutical is produced. When injected into the body this radiopharmaceutical specifically binds to bone, allowing assessment of the skeleton. Similarly, combining technetium-99m with other compounds permits assessment of other parts of the body, for example the urinary tract and cerebral blood flow. Depending on how the radiopharmaceutical is absorbed, distributed, metabolized, and excreted by the body after injection, images are obtained using a gamma camera (Fig. 1.11). Positron emission tomography (PET) is an imaging modality for detecting positron-emitting radionuclides. A positron is an anti-electron, which is a positively charged particle of antimatter. Positrons are emitted from the decay of proton-rich radionuclides. Most of these radionuclides are made in a cyclotron and have extremely short half-lives. The most commonly used PET radionuclide is fluorodeoxyglucose (FDG) labeled with fluorine-18 (a positron emitter). Tissues that are actively metabolizing glucose take up this compound, and the resulting localized high concentration of this molecule compared to background emission is detected as a “hot spot.” PET has become an important imaging modality in the detection of cancer and the assessment of its treatment and recurrence. Single photon emission computed tomography (SPECT) is an imaging modality for detecting gamma rays emitted from the decay of injected radionuclides such as technetium-99m, iodine-123, or iodine-131. The rays are detected by a 360-degree rotating camera, which allows the construction of 3D images. SPECT can be used to diagnose a wide range of disease conditions such as coronary artery disease and bone fractures. Imaging is necessary in most clinical specialties to diagnose pathological changes to tissues. It is paramount to appreciate what is normal and what is abnormal. An appreciation of how the image is obtained, what the normal variations are, and what technical considerations are necessary to obtain a radiological diagnosis. Without understanding the anatomy of the region imaged, it is impossible to comment on the abnormal. Plain radiographs are undoubtedly the most common form of image obtained in a hospital or local practice. Before interpretation, it is important to know about the imaging technique and the views obtained as standard. In most instances (apart from chest radiography) the X-ray tube is 1 m away from the X-ray film. The object in question, for example a hand or a foot, is placed upon the film. When describing subject placement for radiography, the part closest to the X-ray tube is referred to first and that closest to the film is referred to second. For example, when positioning a patient for an anteroposterior (AP) radiograph, the more anterior part of the body is closest to the tube and the posterior part is closest to the film. When X-rays are viewed on a viewing box, the right side of the patient is placed to the observer’s left; therefore, the observer views the radiograph as though looking at a patient in the anatomical position. The chest radiograph is one of the most commonly requested plain radiographs. An image is taken with the patient erect and placed posteroanteriorly (PA chest radiograph; that is, with the patient’s back closest to the X-ray tube.). Occasionally, when patients are too unwell to stand erect, films are obtained on the bed in an anteroposterior (AP) position. These films are less standardized than PA films, and caution should always be taken when interpreting AP radiographs. The plain chest radiograph should always be checked for quality. Film markers should be placed on the appropriate side. (Occasionally patients have dextrocardia, which may be misinterpreted if the film marker is placed inappropriately.) A good-quality chest radiograph will demonstrate the lungs, cardiomediastinal contour, diaphragm, ribs, and peripheral soft tissues. Plain abdominal radiographs are obtained in the AP supine position. From time to time an erect plain abdominal radiograph is obtained when small bowel obstruction is suspected. High-density contrast medium is ingested to opacify the esophagus, stomach, small bowel, and large bowel. As described previously (p. 6), the bowel is insufflated with air (or carbon dioxide) to provide a double-contrast study. In many countries, endoscopy has superseded upper gastrointestinal imaging, but the mainstay of imaging the large bowel is the double-contrast barium enema. Typically the patient needs to undergo bowel preparation, in which powerful cathartics are used to empty the bowel. At the time of the examination a small tube is placed into the rectum and a barium suspension is run into the large bowel. The patient undergoes a series of twists and turns so that the contrast passes through the entire large bowel. The contrast is emptied and air is passed through the same tube to insufflate the large bowel. A thin layer of barium coats the normal mucosa, allowing mucosal detail to be visualized (see Fig. 1.4). Intravenous urography is the standard investigation for assessing the urinary tract. Intravenous contrast medium is injected, and images are obtained as the medium is excreted through the kidneys. A series of films are obtained during this period from immediately after the injection up to approximately 20 minutes later, when the bladder is full of contrast medium. This series of radiographs demonstrates the kidneys, ureters, and bladder and enables assessment of the retroperitoneum and other structures that may press on the urinary tract. Computed tomography is the preferred terminology rather than computerized tomography, though both terms are used interchangeably by physicians. It is important for the student to understand the presentation of images. Most images are acquired in the axial plane and viewed such that the observer looks from below and upward toward the head (from the foot of the bed). By implication: the right side of the patient is on the left side of the image, and the uppermost border of the image is anterior. Many patients are given oral and intravenous contrast media to differentiate bowel loops from other abdominal organs and to assess the vascularity of normal anatomical structures. When intravenous contrast is given, the earlier the images are obtained, the greater the likelihood of arterial enhancement. As the time is delayed between injection and image acquisition, a venous phase and an equilibrium phase are also obtained. The great advantage of CT scanning is the ability to extend and compress the gray scale to visualize the bones, soft tissues, and visceral organs. Altering the window settings and window centering provides the physician with specific information about these structures. There is no doubt that MRI has revolutionized the understanding and interpretation of the brain and its coverings. Furthermore, it has significantly altered the practice of musculoskeletal medicine and surgery. Images can be obtained in any plane and in most sequences. Typically the images are viewed using the same principles as CT. Intravenous contrast agents are also used to further enhance tissue contrast. Typically, MRI contrast agents contain paramagnetic substances (e.g., gadolinium and manganese). Most nuclear medicine images are functional studies. Images are usually interpreted directly from a computer, and a series of representative films are obtained for clinical use. Whenever a patient undergoes an X-ray or nuclear medicine investigation, a dose of radiation is given (Table 1.1). As a general principle it is expected that the dose given is as low as reasonably possible for a diagnostic image to be obtained. Numerous laws govern the amount of radiation exposure that a patient can undergo for a variety of procedures, and these are monitored to prevent any excess or additional dosage. Whenever a radiograph is booked, the clinician ordering the procedure must appreciate its necessity and understand the dose given to the patient to ensure that the benefits significantly outweigh the risks. Imaging modalities such as ultrasound and MRI are ideal because they do not impart significant risk to the patient. Moreover, ultrasound imaging is the modality of choice for assessing the fetus. Any imaging device is expensive, and consequently the more complex the imaging technique (e.g., MRI) the more expensive the investigation. Investigations must be carried out judiciously, based on a sound clinical history and examination, for which an understanding of anatomy is vital. The skeleton can be divided into two subgroups, the axial skeleton and the appendicular skeleton. The axial skeleton consists of the bones of the skull (cranium), vertebral column, ribs, and sternum, whereas the appendicular skeleton consists of the bones of the upper and lower limbs (Fig. 1.12). The skeletal system consists of cartilage and bone. Cartilage is an avascular form of connective tissue consisting of extracellular fibers embedded in a matrix that contains cells localized in small cavities. The amount and kind of extracellular fibers in the matrix varies depending on the type of cartilage. In heavy weightbearing areas or areas prone to pulling forces, the amount of collagen is greatly increased and the cartilage is almost inextensible. In contrast, in areas where weightbearing demands and stress are less, cartilage containing elastic fibers and fewer collagen fibers is common. The functions of cartilage are to: support soft tissues, provide a smooth, gliding surface for bone articulations at joints, and enable the development and growth of long bones. There are three types of cartilage: hyaline—most common; matrix contains a moderate amount of collagen fibers (e.g., articular surfaces of bones); elastic—matrix contains collagen fibers along with a large number of elastic fibers (e.g., external ear); fibrocartilage—matrix contains a limited number of cells and ground substance amidst a substantial amount of collagen fibers (e.g., intervertebral discs). Cartilage is nour"
+    },
+    {
+        "text": "the bones of the upper and lower limbs (Fig. 1.12). The skeletal system consists of cartilage and bone. Cartilage is an avascular form of connective tissue consisting of extracellular fibers embedded in a matrix that contains cells localized in small cavities. The amount and kind of extracellular fibers in the matrix varies depending on the type of cartilage. In heavy weightbearing areas or areas prone to pulling forces, the amount of collagen is greatly increased and the cartilage is almost inextensible. In contrast, in areas where weightbearing demands and stress are less, cartilage containing elastic fibers and fewer collagen fibers is common. The functions of cartilage are to: support soft tissues, provide a smooth, gliding surface for bone articulations at joints, and enable the development and growth of long bones. There are three types of cartilage: hyaline—most common; matrix contains a moderate amount of collagen fibers (e.g., articular surfaces of bones); elastic—matrix contains collagen fibers along with a large number of elastic fibers (e.g., external ear); fibrocartilage—matrix contains a limited number of cells and ground substance amidst a substantial amount of collagen fibers (e.g., intervertebral discs). Cartilage is nourished by diffusion and has no blood vessels, lymphatics, or nerves. Bone is a calcified, living, connective tissue that forms the majority of the skeleton. It consists of an intercellular calcified matrix, which also contains collagen fibers, and several types of cells within the matrix. Bones function as: supportive structures for the body, protectors of vital organs, reservoirs of calcium and phosphorus, levers on which muscles act to produce movement, and containers for blood-producing cells. There are two types of bone, compact and spongy (trabecular or cancellous). Compact bone is dense bone that forms the outer shell of all bones and surrounds spongy bone. Spongy bone consists of spicules of bone enclosing cavities containing blood-forming cells (marrow). Classification of bones is by shape. Long bones are tubular (e.g., humerus in upper limb; femur in lower limb). Short bones are cuboidal (e.g., bones of the wrist and ankle). Flat bones consist of two compact bone plates separated by spongy bone (e.g., skull). Irregular bones are bones with various shapes (e.g., bones of the face). Sesamoid bones are round or oval bones that develop in tendons. Bones are vascular and are innervated. Generally, an adjacent artery gives off a nutrient artery, usually one per bone, that directly enters the internal cavity of the bone and supplies the marrow, spongy bone, and inner layers of compact bone. In addition, all bones are covered externally, except in the area of a joint where articular cartilage is present, by a fibrous connective tissue membrane called the periosteum, which has the unique capability of forming new bone. This membrane receives blood vessels whose branches supply the outer layers of compact bone. A bone stripped of its periosteum will not survive. Nerves accompany the vessels that supply the bone and the periosteum. Most of the nerves passing into the internal cavity with the nutrient artery are vasomotor fibers that regulate blood flow. Bone itself has few sensory nerve fibers. On the other hand, the periosteum is supplied with numerous sensory nerve fibers and is very sensitive to any type of injury. Developmentally, all bones come from mesenchyme by either intramembranous ossification, in which mesenchymal models of bones undergo ossification, or endochondral ossification, in which cartilaginous models of bones form from mesenchyme and undergo ossification. The sites where two skeletal elements come together are termed joints. The two general categories of joints (Fig. 1.18) are those in which: the skeletal elements are separated by a cavity (i.e., synovial joints), and there is no cavity and the components are held together by connective tissue (i.e., solid joints). Blood vessels that cross over a joint and nerves that innervate muscles acting on a joint usually contribute articular branches to that joint. Synovial joints are connections between skeletal components where the elements involved are separated by a narrow articular cavity (Fig. 1.19). In addition to containing an articular cavity, these joints have a number of characteristic features. First, a layer of cartilage, usually hyaline cartilage, covers the articulating surfaces of the skeletal elements. In other words, bony surfaces do not normally contact one another directly. As a consequence, when these joints are viewed in normal radiographs, a wide gap seems to separate the adjacent bones because the cartilage that covers the articulating surfaces is more transparent to X-rays than bone. A second characteristic feature of synovial joints is the presence of a joint capsule consisting of an inner synovial membrane and an outer fibrous membrane. The synovial membrane attaches to the margins of the joint surfaces at the interface between the cartilage and bone and encloses the articular cavity. The synovial membrane is highly vascular and produces synovial fluid, which percolates into the articular cavity and lubricates the articulating surfaces. Closed sacs of synovial membrane also occur outside joints, where they form synovial bursae or tendon sheaths. Bursae often intervene between structures, such as tendons and bone, tendons and joints, or skin and bone, and reduce the friction of one structure moving over the other. Tendon sheaths surround tendons and also reduce friction. The fibrous membrane is formed by dense connective tissue and surrounds and stabilizes the joint. Parts of the fibrous membrane may thicken to form ligaments, which further stabilize the joint. Ligaments outside the capsule usually provide additional reinforcement. Another common but not universal feature of synovial joints is the presence of additional structures within the area enclosed by the capsule or synovial membrane, such as articular discs (usually composed of fibrocartilage), fat pads, and tendons. Articular discs absorb compression forces, adjust to changes in the contours of joint surfaces during movements, and increase the range of movements that can occur at joints. Fat pads usually occur between the synovial membrane and the capsule and move into and out of regions as joint contours change during movement. Redundant regions of the synovial membrane and fibrous membrane allow for large movements at joints. Descriptions of synovial joints based on shape and movement Synovial joints are described based on shape and movement: based on the shape of their articular surfaces, synovial joints are described as plane (flat), hinge, pivot, bicondylar (two sets of contact points), condylar (ellipsoid), saddle, and ball and socket; based on movement, synovial joints are described as uniaxial (movement in one plane), biaxial (movement in two planes), and multiaxial (movement in three planes). Hinge joints are uniaxial, whereas ball and socket joints are multiaxial. Specific types of synovial joints (Fig. 1.20) Plane joints—allow sliding or gliding movements when one bone moves across the surface of another (e.g., acromioclavicular joint) Hinge joints—allow movement around one axis that passes transversely through the joint; permit flexion and extension (e.g., elbow [humero-ulnar] joint) Pivot joints—allow movement around one axis that passes longitudinally along the shaft of the bone; permit rotation (e.g., atlanto-axial joint) Bicondylar joints—allow movement mostly in one axis with limited rotation around a second axis; formed by two convex condyles that articulate with concave or flat surfaces (e.g., knee joint) Condylar (ellipsoid) joints—allow movement around two axes that are at right angles to each other; permit flexion, extension, abduction, adduction, and circumduction (limited) (e.g., wrist joint) Saddle joints—allow movement around two axes that are at right angles to each other; the articular surfaces are saddle shaped; permit flexion, extension, abduction, adduction, and circumduction (e.g., carpometacarpal joint of the thumb) Ball and socket joints—allow movement around multiple axes; permit flexion, extension, abduction, adduction, circumduction, and rotation (e.g., hip Solid joints are connections between skeletal elements where the adjacent surfaces are linked together either by fibrous connective tissue or by cartilage, usually fibrocartilage (Fig. 1.21). Movements at these joints are more restricted than at synovial joints. Fibrous joints include sutures, gomphoses, and syndesmoses. Sutures occur only in the skull where adjacent bones are linked by a thin layer of connective tissue termed a sutural ligament. Gomphoses occur only between the teeth and adjacent bone. In these joints, short collagen tissue fibers in the periodontal ligament run between the root of the tooth and the bony socket. Syndesmoses are joints in which two adjacent bones are linked by a ligament. Examples are the ligamentum flavum, which connects adjacent vertebral laminae, and an interosseous membrane, which links, for example, the radius and ulna in the forearm. Cartilaginous joints include synchondroses and symphyses. Synchondroses occur where two ossification centers in a developing bone remain separated by a layer of cartilage, for example, the growth plate that occurs between the head and shaft of developing long bones. These joints allow bone growth and eventually become"
+    },
+    {
+        "text": "linked together either by fibrous connective tissue or by cartilage, usually fibrocartilage (Fig. 1.21). Movements at these joints are more restricted than at synovial joints. Fibrous joints include sutures, gomphoses, and syndesmoses. Sutures occur only in the skull where adjacent bones are linked by a thin layer of connective tissue termed a sutural ligament. Gomphoses occur only between the teeth and adjacent bone. In these joints, short collagen tissue fibers in the periodontal ligament run between the root of the tooth and the bony socket. Syndesmoses are joints in which two adjacent bones are linked by a ligament. Examples are the ligamentum flavum, which connects adjacent vertebral laminae, and an interosseous membrane, which links, for example, the radius and ulna in the forearm. Cartilaginous joints include synchondroses and symphyses. Synchondroses occur where two ossification centers in a developing bone remain separated by a layer of cartilage, for example, the growth plate that occurs between the head and shaft of developing long bones. These joints allow bone growth and eventually become completely ossified. Symphyses occur where two separate bones are interconnected by cartilage. Most of these types of joints occur in the midline and include the pubic symphysis between the two pelvic bones, and intervertebral discs between adjacent vertebrae. The skin is the largest organ of the body. It consists of the epidermis and the dermis. The epidermis is the outer cellular layer of stratified squamous epithelium, which is avascular and varies in thickness. The dermis is a dense bed of vascular connective tissue. The skin functions as a mechanical and permeability barrier, and as a sensory and thermoregulatory organ. It also can initiate primary immune responses. Fascia is connective tissue containing varying amounts of fat that separate, support, and interconnect organs and structures, enable movement of one structure relative to another, and allow the transit of vessels and nerves from one area to another. There are two general categories of fascia: superficial and deep. Superficial (subcutaneous) fascia lies just deep to and is attached to the dermis of the skin. It is made up of loose connective tissue usually containing a large amount of fat. The thickness of the superficial fascia (subcutaneous tissue) varies considerably, both from one area of the body to another and from one individual to another. The superficial fascia allows movement of the skin over deeper areas of the body, acts as a conduit for vessels and nerves coursing to and from the skin, and serves as an energy (fat) reservoir. Deep fascia usually consists of dense, organized connective tissue. The outer layer of deep fascia is attached to the deep surface of the superficial fascia and forms a thin fibrous covering over most of the deeper region of the body. Inward extensions of this fascial layer form intermuscular septa that compartmentalize groups of muscles with similar functions and innervations. Other extensions surround individual muscles and groups of vessels and nerves, forming an investing fascia. Near some joints the deep fascia thickens, forming retinacula. These fascial retinacula hold tendons in place and prevent them from bowing during movements at the joints. Finally, there is a layer of deep fascia separating the membrane lining the abdominal cavity (the parietal peritoneum) from the fascia covering the deep surface of the muscles of the abdominal wall (the transversalis fascia). This layer is referred to as extraperitoneal fascia. A similar layer of fascia in the thorax is termed the endothoracic fascia. The muscular system is generally regarded as consisting of one type of muscle found in the body—skeletal muscle. However, there are two other types of muscle tissue found in the body, smooth muscle and cardiac muscle, that are important components of other systems. These three types of muscle can be characterized by whether they are controlled voluntarily or involuntarily, whether they appear striated (striped) or smooth, and whether they are associated with the body wall (somatic) or with organs and blood vessels (visceral). Skeletal muscle forms the majority of the muscle tissue in the body. It consists of parallel bundles of long, multinucleated fibers with transverse stripes, is capable of powerful contractions, and is innervated by somatic and branchial motor nerves. This muscle is used to move bones and other structures, and provides support and gives form to the body. Individual skeletal muscles are often named on the basis of shape (e.g., rhomboid major muscle), attachments (e.g., sternohyoid muscle), function (e.g., flexor pollicis longus muscle), position (e.g., palmar interosseous muscle), or fiber orientation (e.g., external oblique muscle). Cardiac muscle is striated muscle found only in the walls of the heart (myocardium) and in some of the large vessels close to where they join the heart. It consists of a branching network of individual cells linked electrically and mechanically to work as a unit. Its contractions are less powerful than those of skeletal muscle and it is resistant to fatigue. Cardiac muscle is innervated by visceral motor nerves. Smooth muscle (absence of stripes) consists of elongated or spindle-shaped fibers capable of slow and sustained contractions. It is found in the walls of blood vessels (tunica media), associated with hair follicles in the skin, located in the eyeball, and found in the walls of various structures associated with the gastrointestinal, respiratory, genitourinary, and urogenital systems. Smooth muscle is innervated by visceral motor nerves. The cardiovascular system consists of the heart, which pumps blood throughout the body, and the blood vessels, which are a closed network of tubes that transport the blood. There are three types of blood vessels: arteries, which transport blood away from the heart; veins, which transport blood toward the heart; capillaries, which connect the arteries and veins, are the smallest of the blood vessels and are where oxygen, nutrients, and wastes are exchanged within the tissues. The walls of the blood vessels of the cardiovascular system usually consist of three layers or tunics: tunica externa (adventitia)—the outer connective tissue layer, tunica media—the middle smooth muscle layer (may also contain varying amounts of elastic fibers in medium and large arteries), and tunica intima—the inner endothelial lining of the blood vessels. Arteries are usually further subdivided into three classes, according to the variable amounts of smooth muscle and elastic fibers contributing to the thickness of the tunica media, the overall size of the vessel, and its function. Large elastic arteries contain substantial amounts of elastic fibers in the tunica media, allowing expansion and recoil during the normal cardiac cycle. This helps maintain a constant flow of blood during diastole. Examples of large elastic arteries are the aorta, the brachiocephalic trunk, the left common carotid artery, the left subclavian artery, and the pulmonary trunk. Medium muscular arteries are composed of a tunica media that contains mostly smooth muscle fibers. This characteristic allows these vessels to regulate their diameter and control the flow of blood to different parts of the body. Examples of medium muscular arteries are most of the named arteries, including the femoral, axillary, and radial arteries. Small arteries and arterioles control the filling of the capillaries and directly contribute to the arterial pressure in the vascular system. Veins also are subdivided into three classes. Large veins contain some smooth muscle in the tunica media, but the thickest layer is the tunica externa. Examples of large veins are the superior vena cava, the inferior vena cava, and the portal vein. Small and medium veins contain small amounts of smooth muscle, and the thickest layer is the tunica externa. Examples of small and medium veins are superficial veins in the upper and lower limbs and deeper veins of the leg and forearm. Venules are the smallest veins and drain the capillaries. Although veins are similar in general structure to arteries, they have a number of distinguishing features. The walls of veins, specifically the tunica media, are thin. The luminal diameters of veins are large. There often are multiple veins (venae comitantes) closely associated with arteries in peripheral regions. Valves often are present in veins, particularly in peripheral vessels inferior to the level of the heart. These are usually paired cusps that facilitate blood flow toward the heart. More specific information about the cardiovascular system and how it relates to the circulation of blood throughout the body will be discussed, where appropriate, in each of the succeeding chapters of the text. Lymphatic vessels form an extensive and complex interconnected network of channels, which begin as “porous” blind-ended lymphatic capillaries in tissues of the body and converge to form a number of larger vessels, which ultimately connect with large veins in the root of the neck. Lymphatic vessels mainly collect fluid lost from vascular capillary beds during nutrient exchange processes and deliver it back to the venous side of the vascular system (Fig. 1.28). Also included in this interstitial fluid that drains into the lymphatic capillaries are pathogens, cells of the lymphocytic system, cell products (such as hormones), and cell debris. In the small intestine, certain fats absorbed and processed by the intestinal epithelium are packaged into protein-coated lipid droplets (chylomicrons), which are released from the epithelial cells and enter the interstitial compartment. Together with other components of the interstitial fluid, the chylomicrons drain into lymphatic capillaries (known as lacte"
+    },
+    {
+        "text": "These are usually paired cusps that facilitate blood flow toward the heart. More specific information about the cardiovascular system and how it relates to the circulation of blood throughout the body will be discussed, where appropriate, in each of the succeeding chapters of the text. Lymphatic vessels form an extensive and complex interconnected network of channels, which begin as “porous” blind-ended lymphatic capillaries in tissues of the body and converge to form a number of larger vessels, which ultimately connect with large veins in the root of the neck. Lymphatic vessels mainly collect fluid lost from vascular capillary beds during nutrient exchange processes and deliver it back to the venous side of the vascular system (Fig. 1.28). Also included in this interstitial fluid that drains into the lymphatic capillaries are pathogens, cells of the lymphocytic system, cell products (such as hormones), and cell debris. In the small intestine, certain fats absorbed and processed by the intestinal epithelium are packaged into protein-coated lipid droplets (chylomicrons), which are released from the epithelial cells and enter the interstitial compartment. Together with other components of the interstitial fluid, the chylomicrons drain into lymphatic capillaries (known as lacteals in the small intestine) and are ultimately delivered to the venous system in the neck. The lymphatic system is therefore also a major route of transport for fat absorbed by the gut. The fluid in most lymphatic vessels is clear and colorless and is known as lymph. That carried by lymphatic vessels from the small intestine is opaque and milky because of the presence of chylomicrons and is termed chyle. There are lymphatic vessels in most areas of the body, including those associated with the central nervous system (Louveau A et al., Nature 2015; 523:337-41; Aspelund A et al., J Exp Med 2015; 212:991-9). Exceptions include bone marrow and avascular tissues such as epithelia and cartilage. The movement of lymph through the lymphatic vessels is generated mainly by the indirect action of adjacent structures, particularly by contraction of skeletal muscles and pulses in arteries. Unidirectional flow is maintained by the presence of valves in the vessels. Lymph nodes are small (0.1–2.5 cm long) encapsulated structures that interrupt the course of lymphatic vessels and contain elements of the body’s defense system, such as clusters of lymphocytes and macrophages. They act as elaborate filters that trap and phagocytose particulate matter in the lymph that percolates through them. In addition, they detect and defend against foreign antigens that are also carried in the lymph (Fig. 1.28). Because lymph nodes are efficient filters and flow through them is slow, cells that metastasize from (migrate away from) primary tumors and enter lymphatic vessels often lodge and grow as secondary tumors in lymph nodes. Lymph nodes that drain regions that are infected or contain other forms of disease can enlarge or undergo certain physical changes, such as becoming “hard” or “tender.” These changes can be used by clinicians to detect pathologic changes or to track spread of disease. A number of regions in the body are associated with clusters or a particular abundance of lymph nodes (Fig. 1.29). Not surprisingly, nodes in many of these regions drain the body’s surface, the digestive system, or the respiratory system. All three of these areas are high-risk sites for the entry of foreign pathogens. Lymph nodes are abundant and accessible to palpation in the axilla, the groin and femoral region, and the neck. Deep sites that are not palpable include those associated with the trachea and bronchi in the thorax, and with the aorta and its branches in the abdomen. All lymphatic vessels coalesce to form larger trunks or ducts, which drain into the venous system at sites in the neck where the internal jugular veins join the subclavian veins to form the brachiocephalic veins (Fig. 1.30): Lymph from the right side of the head and neck, the right upper limb, and the right side of the thorax is carried by lymphatic vessels that connect with veins on the right side of the neck. Lymph from all other regions of the body is carried by lymphatic vessels that drain into veins on the left side of the neck. Specific information about the organization of the lymphatic system in each region of the body is discussed in the appropriate chapter. The nervous system can be separated into parts based on structure and on function: structurally, it can be divided into the central nervous system (CNS) and the peripheral nervous system (PNS) (Fig. 1.32); functionally, it can be divided into somatic and visceral parts. The CNS is composed of the brain and spinal cord, both of which develop from the neural tube in the embryo. The PNS is composed of all nervous structures outside the CNS that connect the CNS to the body. Elements of this system develop from neural crest cells and as outgrowths of the CNS. The PNS consists of the spinal and cranial nerves, visceral nerves and plexuses, and the enteric system. The detailed anatomy of a typical spinal nerve is described in Chapter 2, as is the way spinal nerves are numbered. Cranial nerves are described in Chapter 8. The details of nerve plexuses are described in chapters dealing with the specific regions in which the plexuses are located. The parts of the brain are the cerebral hemispheres, the cerebellum, and the brainstem. The cerebral hemispheres consist of an outer portion, or the gray matter, containing cell bodies; an inner portion, or the white matter, made up of axons forming tracts or pathways; and the ventricles, which are spaces filled with CSF. The cerebellum has two lateral lobes and a midline portion. The components of the brainstem are classically defined as the diencephalon, midbrain, pons, and medulla. However, in common usage today, the term “brainstem” usually refers to the midbrain, pons, and medulla. A further discussion of the brain can be found in Chapter 8. The spinal cord is the part of the CNS in the superior two thirds of the vertebral canal. It is roughly cylindrical in shape, and is circular to oval in cross section with a central canal. A further discussion of the spinal cord can be found in Chapter 2. The meninges (Fig. 1.33) are three connective tissue coverings that surround, protect, and suspend the brain and spinal cord within the cranial cavity and vertebral canal, respectively: The dura mater is the thickest and most external of the coverings. The arachnoid mater is against the internal surface of the dura mater. The pia mater is adherent to the brain and spinal cord. Between the arachnoid and pia mater is the subarachnoid space, which contains CSF. A further discussion of the cranial meninges can be found in Chapter 8 and of the spinal meninges in Chapter 2. Functional subdivisions of the CNS Functionally, the nervous system can be divided into somatic and visceral parts. The somatic part (soma, from the Greek for “body”) innervates structures (skin and most skeletal muscle) derived from somites in the embryo, and is mainly involved with receiving and responding to information from the external environment. The visceral part (viscera, from the Greek for “guts”) innervates organ systems in the body and other visceral elements, such as smooth muscle and glands, in peripheral regions of the body. It is concerned mainly with detecting and responding to information from the internal environment. Somatic part of the nervous system The somatic part of the nervous system consists of: nerves that carry conscious sensations from peripheral regions back to the CNS, and nerves that innervate voluntary muscles. Somatic nerves arise segmentally along the developing CNS in association with somites, which are themselves arranged segmentally along each side of the neural tube (Fig. 1.34). Part of each somite (the dermatomyotome) gives rise to skeletal muscle and the dermis of the skin. As cells of the dermatomyotome differentiate, they migrate into posterior (dorsal) and anterior (ventral) areas of the developing body: Cells that migrate anteriorly give rise to muscles of the limbs and trunk (hypaxial muscles) and to the associated dermis. Cells that migrate posteriorly give rise to the intrinsic muscles of the back (epaxial muscles) and the associated dermis. Developing nerve cells within anterior regions of the neural tube extend processes peripherally into posterior and anterior regions of the differentiating dermatomyotome of each somite. Simultaneously, derivatives of neural crest cells (cells derived from neural folds during formation of the neural tube) differentiate into neurons on each side of the neural tube and extend processes both medially and laterally (Fig. 1.35): Medial processes pass into the posterior aspect of the neural tube. Lateral processes pass into the differentiating regions of the adjacent dermatomyotome. Neurons that develop from cells within the spinal cord are motor neurons and those that develop from neural crest cells are sensory neurons. Somatic sensory and somatic motor fibers that are organized segmentally along the neural tube become parts of all spinal nerves and some cranial nerves. The clusters of sensory nerve cell bodies derived from neural crest cells and located outside the CNS form sensory ganglia. Generally, all sensory information passes into the posterior aspect of the spinal cord, and all motor fibers leave anteriorly."
+    },
+    {
+        "text": "the limbs and trunk (hypaxial muscles) and to the associated dermis. Cells that migrate posteriorly give rise to the intrinsic muscles of the back (epaxial muscles) and the associated dermis. Developing nerve cells within anterior regions of the neural tube extend processes peripherally into posterior and anterior regions of the differentiating dermatomyotome of each somite. Simultaneously, derivatives of neural crest cells (cells derived from neural folds during formation of the neural tube) differentiate into neurons on each side of the neural tube and extend processes both medially and laterally (Fig. 1.35): Medial processes pass into the posterior aspect of the neural tube. Lateral processes pass into the differentiating regions of the adjacent dermatomyotome. Neurons that develop from cells within the spinal cord are motor neurons and those that develop from neural crest cells are sensory neurons. Somatic sensory and somatic motor fibers that are organized segmentally along the neural tube become parts of all spinal nerves and some cranial nerves. The clusters of sensory nerve cell bodies derived from neural crest cells and located outside the CNS form sensory ganglia. Generally, all sensory information passes into the posterior aspect of the spinal cord, and all motor fibers leave anteriorly. Somatic sensory neurons carry information from the periphery into the CNS and are also called somatic sensory afferents or general somatic afferents (GSAs). The modalities carried by these nerves include temperature, pain, touch, and proprioception. Proprioception is the sense of determining the position and movement of the musculoskeletal system detected by special receptors in muscles and tendons. Somatic motor fibers carry information away from the CNS to skeletal muscles and are also called somatic motor efferents or general somatic efferents (GSEs). Like somatic sensory fibers that come from the periphery, somatic motor fibers can be very long. They extend from cell bodies in the spinal cord to the muscle cells they innervate. Because cells from a specific somite develop into the dermis of the skin in a precise location, somatic sensory fibers originally associated with that somite enter the posterior region of the spinal cord at a specific level and become part of one specific spinal nerve (Fig. 1.36). Each spinal nerve therefore carries somatic sensory information from a specific area of skin on the surface of the body. A dermatome is that area of skin supplied by a single spinal cord level, or on one side, by a single spinal nerve. There is overlap in the distribution of dermatomes, but usually a specific region within each dermatome can be identified as an area supplied by a single spinal cord level. Testing touch in these autonomous zones in a conscious patient can be used to localize lesions to a specific spinal nerve or to a specific level in the spinal cord. Somatic motor nerves that were originally associated with a specific somite emerge from the anterior region of the spinal cord and, together with sensory nerves from the same level, become part of one spinal nerve. Therefore each spinal nerve carries somatic motor fibers to muscles that originally developed from the related somite. A myotome is that portion of a skeletal muscle innervated by a single spinal cord level or, on one side, by a single spinal nerve. Myotomes are generally more difficult to test than dermatomes because each skeletal muscle in the body often develops from more than one somite and is therefore innervated by nerves derived from more than one spinal cord level (Fig. 1.37). Testing movements at successive joints can help in localizing lesions to specific nerves or to a specific spinal cord level. For example: Muscles that move the shoulder joint are innervated mainly by spinal nerves from spinal cord levels C5 and C6. Muscles that move the elbow are innervated mainly by spinal nerves from spinal cord levels C6 and C7. Muscles in the hand are innervated mainly by spinal nerves from spinal cord levels C8 and T1. Visceral part of the nervous system The visceral part of the nervous system, as in the somatic part, consists of motor and sensory components: Sensory nerves monitor changes in the viscera. Motor nerves mainly innervate smooth muscle, cardiac muscle, and glands. The visceral motor component is commonly referred to as the autonomic division of the PNS and is subdivided into sympathetic and parasympathetic parts. Like the somatic part of the nervous system, the visceral part is segmentally arranged and develops in a parallel fashion (Fig. 1.39). Visceral sensory neurons that arise from neural crest cells send processes medially into the adjacent neural tube and laterally into regions associated with the developing body. These sensory neurons and their processes, referred to as general visceral afferent fibers (GVAs), are associated primarily with chemoreception, mechanoreception, and stretch reception. Visceral motor neurons that arise from cells in lateral regions of the neural tube send processes out of the anterior aspect of the tube. Unlike in the somatic part, these processes, containing general visceral efferent fibers (GVEs), synapse with other cells, usually other visceral motor neurons, that develop outside the CNS from neural crest cells that migrate away from their original positions close to the developing neural tube. The visceral motor neurons located in the spinal cord are referred to as preganglionic motor neurons and their axons are called preganglionic fibers; the visceral motor neurons located outside the CNS are referred to as postganglionic motor neurons and their axons are called postganglionic fibers. The cell bodies of the visceral motor neurons outside the CNS often associate with each other in a discrete mass called a ganglion. Visceral sensory and motor fibers enter and leave the CNS with their somatic equivalents (Fig. 1.40). Visceral sensory fibers enter the spinal cord together with somatic sensory fibers through posterior roots of spinal nerves. Preganglionic fibers of visceral motor neurons exit the spinal cord in the anterior roots of spinal nerves, along with fibers from somatic motor neurons. Postganglionic fibers traveling to visceral elements in the periphery are found in the posterior and anterior rami (branches) of spinal nerves. Visceral motor and sensory fibers that travel to and from viscera form named visceral branches that are separate from the somatic branches. These nerves generally form plexuses from which arise branches to the viscera. Visceral motor and sensory fibers do not enter and leave the CNS at all levels (Fig. 1.41): In the cranial region, visceral components are associated with four of the twelve cranial nerves (CN III, VII, IX, and X). In the spinal cord, visceral components are associated mainly with spinal cord levels T1 to L2 and S2 to S4. Visceral motor components associated with spinal levels T1 to L2 are termed sympathetic. Those visceral motor components in cranial and sacral regions, on either side of the sympathetic region, are termed parasympathetic: The sympathetic system innervates structures in peripheral regions of the body and viscera. The parasympathetic system is more restricted to innervation of the viscera only. Spinal sympathetic and spinal parasympathetic neurons share certain developmental and phenotypic features that are different from those of cranial parasympathetic neurons. Based on this, some researchers have suggested reclassifying all spinal visceral motor neurons as sympathetic (Espinosa-Medina I et al. Science 2016;354:893-897). Others are against reclassification, arguing that the results only indicate that the neurons are spinal in origin (Neuhuber W et al. Anat Rec 2017;300:1369-1370). In addition, sacral nerves do not enter the sympathetic trunk, nor do they have postganglionic fibers that travel to the periphery on spinal nerves, as do T1-L2 visceral motor fibers. We have chosen to retain the classification of S2,3,4 visceral motor neurons as parasympathetic. “Parasympathetic” simply means on either side of the “sympathetic,” which correctly describes their anatomy. The sympathetic part of the autonomic division of the PNS leaves thoracolumbar regions of the spinal cord with the somatic components of spinal nerves T1 to L2 (Fig. 1.42). On each side, a paravertebral sympathetic trunk extends from the base of the skull to the inferior end of the vertebral column where the two trunks converge anteriorly to the coccyx at the ganglion impar. Each trunk is attached to the anterior rami of spinal nerves and becomes the route by which sympathetics are distributed to the periphery and all viscera. Visceral motor preganglionic fibers leave the T1 to L2 part of the spinal cord in anterior roots. The fibers then enter the spinal nerves, pass through the anterior rami and into the sympathetic trunks. One trunk is located on each side of the vertebral column (paravertebral) and positioned anterior to the anterior rami. Along the trunk is a series of segmentally arranged ganglia formed from collections of postganglionic neuronal cell bodies where the preganglionic neurons synapse with postganglionic neurons. Anterior rami of T1 to L2 are connected to the sympathetic trunk or to a ganglion by a white ramus communicans, which carries preganglionic sympathetic fibers and appears white because the fibers it contains are myelinated. Preganglionic sympathetic fibers that enter a paravertebral ganglion or the sympathetic trunk through a white ramus communicans may take the following four pathways to target tissues: 1. Peripheral sympathetic innervation at"
+    },
+    {
+        "text": "unks converge anteriorly to the coccyx at the ganglion impar. Each trunk is attached to the anterior rami of spinal nerves and becomes the route by which sympathetics are distributed to the periphery and all viscera. Visceral motor preganglionic fibers leave the T1 to L2 part of the spinal cord in anterior roots. The fibers then enter the spinal nerves, pass through the anterior rami and into the sympathetic trunks. One trunk is located on each side of the vertebral column (paravertebral) and positioned anterior to the anterior rami. Along the trunk is a series of segmentally arranged ganglia formed from collections of postganglionic neuronal cell bodies where the preganglionic neurons synapse with postganglionic neurons. Anterior rami of T1 to L2 are connected to the sympathetic trunk or to a ganglion by a white ramus communicans, which carries preganglionic sympathetic fibers and appears white because the fibers it contains are myelinated. Preganglionic sympathetic fibers that enter a paravertebral ganglion or the sympathetic trunk through a white ramus communicans may take the following four pathways to target tissues: 1. Peripheral sympathetic innervation at the level of origin of the preganglionic fiber Preganglionic sympathetic fibers may synapse with postganglionic motor neurons in ganglia associated with the sympathetic trunk, after which postganglionic fibers enter the same anterior ramus and are distributed with peripheral branches of the posterior and anterior rami of that spinal nerve (Fig. 1.43). The fibers innervate structures at the periphery of the body in regions supplied by the spinal nerve. The gray ramus communicans connects the sympathetic trunk or a ganglion to the anterior ramus and contains the postganglionic sympathetic fibers. It appears gray because postganglionic fibers are nonmyelinated. The gray ramus communicans is positioned medial to the white ramus communicans. 2. Peripheral sympathetic innervation above or below the level of origin of the preganglionic fiber Preganglionic sympathetic fibers may ascend or descend to other vertebral levels where they synapse in ganglia associated with spinal nerves that may or may not have visceral motor input directly from the spinal cord (i.e., those nerves other than T1 to L2) (Fig. 1.44). The postganglionic fibers leave the distant ganglia via gray rami communicantes and are distributed along the posterior and anterior rami of the spinal nerves. The ascending and descending fibers, together with all the ganglia, form the paravertebral sympathetic trunk, which extends the entire length of the vertebral column. The formation of this trunk, on each side, enables visceral motor fibers of the sympathetic part of the autonomic division of the PNS, which ultimately emerge from only a small region of the spinal cord (T1 to L2), to be distributed to peripheral regions innervated by all spinal nerves. White rami communicantes only occur in association with spinal nerves T1 to L2, whereas gray rami communicantes are associated with all spinal nerves. Fibers from spinal cord levels T1 to T5 pass predominantly superiorly, whereas fibers from T5 to L2 pass inferiorly. All sympathetics passing into the head have preganglionic fibers that emerge from spinal cord level T1 and ascend in the sympathetic trunks to the highest ganglion in the neck (the superior cervical ganglion), where they synapse. Postganglionic fibers then travel along blood vessels to target tissues in the head, including blood vessels, sweat glands, small smooth muscles associated with the upper eyelids, and the dilator of the pupil. 3. Sympathetic innervation of thoracic Preganglionic sympathetic fibers may synapse with postganglionic motor neurons in ganglia and then leave the ganglia medially to innervate thoracic or cervical viscera (Fig. 1.45). They may ascend in the trunk before synapsing, and after synapsing the postganglionic fibers may combine with those from other levels to form named visceral nerves, such as cardiac nerves. Often, these nerves join branches from the parasympathetic system to form plexuses on or near the surface of the target organ, for example, the cardiac and pulmonary plexuses. Branches of the plexus innervate the organ. Spinal cord levels T1 to T5 mainly innervate cranial, cervical, and thoracic viscera. 4. Sympathetic innervation of the abdomen and pelvic regions and the adrenals Preganglionic sympathetic fibers may pass through the sympathetic trunk and paravertebral ganglia without synapsing and, together with similar fibers from other levels, form splanchnic nerves (greater, lesser, least, lumbar, and sacral), which pass into the abdomen and pelvic regions (Fig. 1.46). The preganglionic fibers in these nerves are derived from spinal cord levels T5 to L2. The splanchnic nerves generally connect with sympathetic ganglia around the roots of major arteries that branch from the abdominal aorta. These ganglia are part of a large prevertebral plexus that also has input from the parasympathetic part of the autonomic division of the PNS. Postganglionic sympathetic fibers are distributed in extensions of this plexus, predominantly along arteries, to viscera in the abdomen and pelvis. Some of the preganglionic fibers in the prevertebral plexus do not synapse in the sympathetic ganglia of the plexus but pass through the system to the adrenal gland, where they synapse directly with cells of the adrenal medulla. These cells are homologues of sympathetic postganglionic neurons and secrete adrenaline and noradrenaline into the vascular system. The parasympathetic part of the autonomic division of the PNS (Fig. 1.47) leaves cranial and sacral regions of the CNS in association with: cranial nerves III, VII, IX, and X: III, VII, and IX carry parasympathetic fibers to structures within the head and neck only, whereas X (the vagus spinal nerves S2 to S4: sacral parasympathetic fibers innervate inferior abdominal viscera, pelvic viscera, and the arteries associated with erectile tissues of the perineum. Like the visceral motor nerves of the sympathetic part, the visceral motor nerves of the parasympathetic part generally have two neurons in the pathway. The preganglionic neurons are in the CNS, and fibers leave in the cranial nerves. In the sacral region, the preganglionic parasympathetic fibers form special visceral nerves (the pelvic splanchnic nerves), which originate from the anterior rami of S2 to S4 and enter pelvic extensions of the large prevertebral plexus formed around the abdominal aorta. These fibers are distributed to pelvic and abdominal viscera mainly along blood vessels. The postganglionic motor neurons are in the walls of the viscera. In organs of the gastrointestinal system, preganglionic fibers do not have a postganglionic parasympathetic motor neuron in the pathway; instead, preganglionic fibers synapse directly on neurons in the ganglia of the enteric system. The preganglionic parasympathetic motor fibers in CN III, VII, and IX separate from the nerves and connect with one of four distinct ganglia, which house postganglionic motor neurons. These four ganglia are near major branches of CN V. Postganglionic fibers leave the ganglia, join the branches of CN V, and are carried to target tissues (salivary, mucous, and lacrimal glands; constrictor muscle of the pupil; and ciliary muscle in the eye) with these branches. The vagus nerve [X] gives rise to visceral branches along its course. These branches contribute to plexuses associated with thoracic viscera or to the large prevertebral plexus in the abdomen and pelvis. Many of these plexuses also contain sympathetic fibers. When present, postganglionic parasympathetic neurons are in the walls of the target viscera. motor fibers. Visceral sensory fibers follow the course of sympathetic fibers entering the spinal cord at similar spinal cord levels. However, visceral sensory fibers may also enter the spinal cord at levels other than those associated with motor output. For example, visceral sensory fibers from the heart may enter at levels higher than spinal cord level T1. Visceral sensory fibers that accompany sympathetic fibers are mainly concerned with detecting pain. Visceral sensory fibers accompanying parasympathetic fibers are carried mainly in IX and X and in spinal nerves S2 to S4. Visceral sensory fibers in IX carry information from chemoreceptors and baroreceptors associated with the walls of major arteries in the neck, and from receptors in the pharynx. Visceral sensory fibers in X include those from cervical viscera, and major vessels and viscera in the thorax and abdomen. Visceral sensory fibers from pelvic viscera and the distal parts of the colon are carried in S2 to S4. Visceral sensory fibers associated with parasympathetic fibers primarily relay information to the CNS about the status of normal physiological processes and reflex activities. The enteric system The enteric nervous system consists of motor and sensory neurons and their support cells, which form two interconnected plexuses, the myenteric and submucous nerve plexuses, within the walls of the gastrointestinal tract (Fig. 1.48). Each of these plex"
+    },
+    {
+        "text": "those associated with motor output. For example, visceral sensory fibers from the heart may enter at levels higher than spinal cord level T1. Visceral sensory fibers that accompany sympathetic fibers are mainly concerned with detecting pain. Visceral sensory fibers accompanying parasympathetic fibers are carried mainly in IX and X and in spinal nerves S2 to S4. Visceral sensory fibers in IX carry information from chemoreceptors and baroreceptors associated with the walls of major arteries in the neck, and from receptors in the pharynx. Visceral sensory fibers in X include those from cervical viscera, and major vessels and viscera in the thorax and abdomen. Visceral sensory fibers from pelvic viscera and the distal parts of the colon are carried in S2 to S4. Visceral sensory fibers associated with parasympathetic fibers primarily relay information to the CNS about the status of normal physiological processes and reflex activities. The enteric system The enteric nervous system consists of motor and sensory neurons and their support cells, which form two interconnected plexuses, the myenteric and submucous nerve plexuses, within the walls of the gastrointestinal tract (Fig. 1.48). Each of these plexuses is formed by: ganglia, which house the nerve cell bodies and associated cells, and bundles of nerve fibers, which pass between ganglia and from the ganglia into surrounding tissues. Neurons in the enteric system are derived from neural crest cells originally associated with occipitocervical and sacral regions. Interestingly, more neurons are reported to be in the enteric system than in the spinal cord itself. Sensory and motor neurons within the enteric system control reflex activity within and between parts of the gastrointestinal system. These reflexes regulate peristalsis, secretomotor activity, and vascular tone. These activities can occur independently of the brain and spinal cord, but can also be modified by input from preganglionic parasympathetic and postganglionic sympathetic fibers. Sensory information from the enteric system is carried back to the CNS by visceral sensory fibers. Nerve plexuses are either somatic or visceral and combine fibers from different sources or levels to form new nerves with specific targets or destinations (Fig. 1.49). Plexuses of the enteric system also generate reflex activity independent of the CNS. Major somatic plexuses formed from the anterior rami of spinal nerves are the cervical (C1 to C4), brachial (C5 to T1), lumbar (L1 to L4), sacral (L4 to S4), and coccygeal (S5 to Co) plexuses. Except for spinal nerve T1, the anterior rami of thoracic spinal nerves remain independent and do not participate in plexuses. Visceral nerve plexuses are formed in association with viscera and generally contain efferent (sympathetic and parasympathetic) and afferent components (Fig. 1.49). These plexuses include cardiac and pulmonary plexuses in the thorax and a large prevertebral plexus in the abdomen anterior to the aorta, which extends inferiorly onto the lateral walls of the pelvis. The massive prevertebral plexus supplies input to and receives output from all abdominal and pelvic viscera. Specific information about the organization and components of the respiratory, gastrointestinal, and urogenital systems will be discussed in each of the succeeding chapters of this text. Fig. 1.1 The anatomical position, planes, and terms of location and orientation. Feet togethertoes forwardHands by sidespalms forwardFace looking forwardInferior margin of orbit level withtop of external auditory meatusSagittal planeCoronal planeSuperiorAnteriorPosteriorMedialLateralInferiorTransverse, horizontal,or axial plane Fig. 1.2 Cathode ray tube for the production of X-rays. Fig. 1.3 Fluoroscopy unit. Fig. 1.4 Barium sulfate follow-through. Fig. 1.5 Digital subtraction angiogram. Fig. 1.6 Ultrasound examination of the abdomen. Fig. 1.7 Computed tomography scanner. Fig. 1.8 Computed tomography scan of the abdomen at vertebral level L2. Fig. 1.9 A T2-weighted MR image in the sagittal plane of the pelvic viscera in a woman. Fig. 1.10 T1-weighted (A) and T2-weighted (B) MR images of the brain in the coronal plane. Fig. 1.11 A gamma camera. Fig. 1.12 The axial skeleton and the appendicular skeleton. Fig. 1.13 Accessory and sesamoid bones. A. Radiograph of the ankle region showing an accessory bone (os trigonum). B. Radiograph of the feet showing numerous sesamoid bones and an accessory bone (os naviculare). Fig. 1.14 A developmental series of radiographs showing the progressive ossification of carpal (wrist) bones from 3 (A) to 10 (D) years of age. Fig. 1.15 T1-weighted image in the coronal plane, demonstrating the relatively high signal intensity returned from the femoral heads and proximal femoral necks, consistent with yellow marrow. In this young patient, the vertebral bodies return an intermediate darker signal that represents red marrow. There is relatively little fat in these vertebrae; hence the lower signal return. Fig. 1.16 Radiograph, lateral view, showing fracture of the ulna at the elbow joint (A) and repair of this fracture (B) using internal fixation with a plate and multiple screws. Fig. 1.17 Image of the hip joints demonstrating loss of height of the right femoral head with juxta-articular bony sclerosis and subchondral cyst formation secondary to avascular necrosis. There is also significant wasting of the muscles supporting the hip, which is secondary to disuse and pain. Normal left hipBladderAvascular necrosisWasting of gluteal muscle Fig. 1.18 Joints. A. Synovial joint. B. Solid joint. Fig. 1.19 Synovial joints. A. Major features of a synovial joint. B. Accessory structures associated with synovial joints. Fig. 1.20 Various types of synovial joints. A. Condylar (wrist). B. Gliding (radio-ulnar). C. Hinge (elbow). D. Ball and socket (hip). E. Saddle (carpometacarpal of thumb). F. Pivot (atlanto-axial). Fig. 1.21 Solid joints. Fig. 1.22 This operative photograph demonstrates the focal areas of cartilage loss in the patella and femoral condyles throughout the knee joint. Fig. 1.23 This radiograph demonstrates the loss of joint space in the medial compartment and presence of small spiky osteophytic regions at the medial lateral aspect of the joint. OsteophytesLoss of joint space Fig. 1.24 After knee replacement. This radiograph shows the position of the prosthesis. Fig. 1.25 This is a radiograph, anteroposterior view, of the pelvis after a right total hip replacement. There are additional significant degenerative changes in the left hip joint, which will also need to be replaced. Fig. 1.26 Axial inversion recovery MR imaging series, which suppresses fat and soft tissue and leaves high signal intensity where fluid is seen. A muscle tear in the right adductor longus with edema in and around the muscle is shown. Fig. 1.27 Photograph demonstrating varicose veins. Fig. 1.28 Lymphatic vessels mainly collect fluid lost from vascular capillary beds during nutrient exchange processes and deliver it back to the venous side of the vascular system. Fig. 1.29 Regions associated with clusters or a particular abundance of lymph nodes. Cervical nodes(along courseof internaljugular vein)Axillary nodes(in axilla)Deep nodes(related to aortaand celiac trunkand superior andinferior mesentericarteries)Pericranial ring(base of head)Tracheal nodes(nodes related totrachea and bronchi)Inguinal nodes(along course ofinguinal ligament)Femoral nodes(along femoral vein) Fig. 1.30 Major lymphatic vessels that drain into large veins in the neck. Fig. 1.31 A. This computed tomogram with contrast, in the axial plane, demonstrates the normal common carotid arteries and internal jugular veins with numerous other nonenhancing nodules that represent lymph nodes in a patient with lymphoma. B. This computed tomogram with contrast, in the axial plane, demonstrates a large anterior soft tissue mediastinal mass that represents a lymphoma. Fig. 1.32 CNS and PNS. Fig. 1.33 Arrangement of meninges in the cranial cavity. Fig. 1.34 Differentiation of somites in a “tubular” embryo. Fig. 1.35 Somatic sensory and"
+    },
+    {
+        "text": "ular vein)Axillary nodes(in axilla)Deep nodes(related to aortaand celiac trunkand superior andinferior mesentericarteries)Pericranial ring(base of head)Tracheal nodes(nodes related totrachea and bronchi)Inguinal nodes(along course ofinguinal ligament)Femoral nodes(along femoral vein) Fig. 1.30 Major lymphatic vessels that drain into large veins in the neck. Fig. 1.31 A. This computed tomogram with contrast, in the axial plane, demonstrates the normal common carotid arteries and internal jugular veins with numerous other nonenhancing nodules that represent lymph nodes in a patient with lymphoma. B. This computed tomogram with contrast, in the axial plane, demonstrates a large anterior soft tissue mediastinal mass that represents a lymphoma. Fig. 1.32 CNS and PNS. Fig. 1.33 Arrangement of meninges in the cranial cavity. Fig. 1.34 Differentiation of somites in a “tubular” embryo. Fig. 1.35 Somatic sensory and motor neurons. Blue lines indicate motor nerves and red lines indicate sensory nerves. Somatic sensory neurondeveloping from neural crest cellsEpaxial (back) musclesHypaxial musclesAxon of motor neuronprojects to muscle developingfrom dermatomyotomeSomatic motor neuroncell body in anterior regionof neural tube Fig. 1.36 Dermatomes. C6 segment of spinal cordSpinal ganglionDermatomyotomeAutonomous region(where overlap ofdermatomes isleast likely)of C6 dermatome(pad of thumb)Skin on the lateral side of the forearm and on thethumb is innervated by C6 spinal level (spinal nerve).The dermis of the skin in this region develops from the somiteinitially associated with the C6 level of the developing spinal cordCaudalCranialSomite Fig. 1.37 Myotomes. C6 segment of spinal cordMuscles that abduct the arm are innervated by C5 and C6 spinal levels (spinal nerves) and develop from somites initially associated with C5 and C6 regions of developing spinal cordC5 segment of spinal cordDermatomyotomeSomite Fig. 1.38 Dermatomes. A. Anterior view. B. Posterior view. Fig. 1.39 Development of the visceral part of the nervous system. Motor nerve endingassociated withblood vessels,sweat glands,arrector pili musclesat peripheryPart of neural crest developinginto spinal gangliaVisceral motor ganglionMotor nerve ending associated with visceraDeveloping gastrointestinal tractSensory nerve endingBody cavity(coelom)Visceral sensory neuron developsfrom neural crest and becomespart of spinal ganglionVisceral motorpreganglionicneuron in lateralregion of CNS(spinal cord)Postganglionic motor neuron is outside CNS.An aggregation of postganglionic neuronal cellbodies forms a peripheral visceral motor ganglion. Fig. 1.40 Basic anatomy of a thoracic spinal nerve. Fig. 1.41 Parts of the CNS associated with visceral motor components. SympatheticT1 to L2spinal segmentsBrainstemcranial nervesIII, VII, IX, XS2 to S4spinal segmentsParasympathetic Fig. 1.42 Sympathetic part of the autonomic division of the PNS. Abdominal visceraHeartOrgansPeripheralSympathetic nerves followsomatic nerves to periphery(glands, smooth muscle)Pelvic visceraGanglion imparEsophageal plexusPrevertebral plexus Fig. 1.43 Course of sympathetic fibers that travel to the periphery in the same spinal nerves in which they travel out of the spinal cord. Gray ramus communicansT10 spinal nervePosteriorramusAnteriorramusPeripheral distribution of sympatheticscarried peripherally by terminal cutaneousbranches of spinal nerve T1 to L2Motor nerve to sweat glands,smooth muscle of bloodvessels, and arrector pilimuscles in the part of T10dermatome supplied by theanterior ramusT10 spinal segmentWhite ramus communicans Fig. 1.44 Course of sympathetic nerves that travel to the periphery in spinal nerves that are not the ones through which they left the spinal cord. Sympathetic paravertebral trunksPeripheral distribution ofascending sympatheticsPeripheral distribution ofdescending sympathetics(C1) C2 to C8T1 to L2L3 to CoWhite ramus communicansGray ramus communicansPosterior rootGray ramus communicansGray ramus communicansAnterior root Fig. 1.45 Course of sympathetic nerves traveling to the heart. Sympathetic cardiac nervesSympathetic cardiac nervesSympathetic trunkCardiac plexusT1 to T4CervicalWhite ramuscommunicansGray ramuscommunicans Fig. 1.46 Course of sympathetic nerves traveling to abdominal and pelvic viscera. White ramus communicansGray ramus communicansSacral splanchnic nervesLumbar splanchnic nervesLeast splanchnic nervesLesser splanchnic nervesGreater splanchnic nervesPrevertebral plexusand gangliaParavertebralsympathetic trunkAbdominalandpelvic visceraAortaT5 to T9T12T9 to T10(T10 to T11)L1 to L2 Fig. 1.47 Parasympathetic part of the autonomic division of the PNS. Thoracic visceral plexusPrevertebral plexusAbdominal visceraSynapse with nerve cellsof enteric systemErectile tissues of penisand clitorisS2 to S4Sacral parasympatheticoutflow via pelvicsplanchnic nervesCranial parasympatheticoutflow via cranial nervesHeartSubmandibularganglionPterygopalatineganglionOtic ganglionCiliary ganglion[III][VII][IX][X]Pelvic visceraPupillary constrictionTransition from supply by [X]to pelvic splanchnic nervesSalivary glandsLacrimal glandParotid gland Fig. 1.48 Enteric part of the nervous system. Fig. 1.49 Nerve plexuses. C7C6C5C4C3C2C1T1T2T3T4T5T6T7T8T9T10T11T12L1S1S2S3S4S5L2L3L4L5C8GreaterLeastLesserSOMATIC PLEXUSESVISCERAL PLEXUSESCervical plexusanterior rami C1 to C4Brachial plexusanterior rami C5 to T1Lumbar plexusanterior rami L1 to L4Sacral plexusanterior ramiL4 to S4Parasympathetic [X]S2 to S4 pelvic splanchnic nerves(parasympathetic)Pulmonary branchPulmonary branchesCardiac branchesCardiac plexusThoracic aortic plexusEsophageal plexusPrevertebral plexusVagal trunkGanglion imparSacral splanchnic nervesSplanchnicnervesLumbar splanchnicnerves Fig. 1.50 Mechanism for referred pain from an inflamed appendix to the T10 dermatome. Table 1.1 The approximate dosage of radiation exposure as an order of magnitude In the clinic These are extra bones that are not usually found as part of the normal skeleton, but can exist as a normal variant in many people. They are typically found in multiple locations in the wrist and hands, ankles and feet (Fig. 1.13). These should not be mistaken for fractures on imaging. Sesamoid bones are embedded within tendons, the largest of which is the patella. There are many other sesamoids in the body particularly in tendons of the hands and feet, and most frequently in flexor tendons of the thumb and big toe. Degenerative and inflammatory changes of, as well as mechanical stresses on, the accessory bones and sesamoids can cause pain, which can be treated with physiotherapy and targeted steroid injections, but in some severe cases it may be necessary to surgically remove the bone. In the clinic Determination of skeletal age Throughout life the bones develop in a predictable way to form the skeletally mature adult at the end of puberty. In western countries skeletal maturity tends to occur between the ages of 20 and 25 years. However, this may well vary according to geography and socioeconomic conditions. Skeletal maturity will also be determined by genetic factors and disease states. Up until the age of skeletal maturity, bony growth and development follows a typically predictable ordered state, which can be measured through either ultrasound, plain radiographs, or MRI scanning. Typically, the nondominant (left) hand is radiographed, and the radiograph is compared to a series of standard radiographs. From these images the bone age can be determined (Fig. 1.14). In certain disease states, such as malnutrition and hypothyroidism,"
+    }
+]
\ No newline at end of file

dataflow/logger.py

+import os
+import sys
+import logging
+import colorlog
+
+# ---------- 1) 自定义 SUCCESS 等级 ----------
+SUCCESS_LEVEL_NUM = 25
+logging.addLevelName(SUCCESS_LEVEL_NUM, "SUCCESS")
+
+def success(self, message, *args, **kwargs):
+    if self.isEnabledFor(SUCCESS_LEVEL_NUM):
+        self._log(SUCCESS_LEVEL_NUM, message, args, **kwargs)
+
+logging.Logger.success = success
+
+# ---------- 2) stdout / stderr 分流的过滤器 ----------
+class MaxLevelFilter(logging.Filter):
+    """仅放行 level < max_level 的日志（用于 stdout）。"""
+    def __init__(self, max_level):
+        super().__init__()
+        self.max_level = max_level
+    def filter(self, record: logging.LogRecord) -> bool:
+        return record.levelno < self.max_level
+
+class MinLevelFilter(logging.Filter):
+    """仅放行 level >= min_level 的日志（用于 stderr）。"""
+    def __init__(self, min_level):
+        super().__init__()
+        self.min_level = min_level
+    def filter(self, record: logging.LogRecord) -> bool:
+        return record.levelno >= self.min_level
+
+# ---------- 3) 贴近 loguru 的 ColoredFormatter ----------
+def _make_colored_formatter():
+    # 颜色映射（loguru 的感觉：DEBUG/INFO 偏冷色，WARNING 黄，ERROR/CRITICAL 红，SUCCESS 绿）
+    log_colors = {
+        "DEBUG":    "blue",
+        "INFO":     "white",
+        "SUCCESS":  "green",
+        "WARNING":  "yellow",
+        "ERROR":    "red",
+        "CRITICAL": "red,bg_white",
+    }
+    # 二级着色：让 levelname 与 message 按级别着色；name/func/line 用青色
+    secondary = {
+        "levelname": log_colors,
+        "message":   {
+            "DEBUG":    "blue",
+            "INFO":     "white",
+            "SUCCESS":  "green",
+            "WARNING":  "yellow",
+            "ERROR":    "red",
+            "CRITICAL": "red",
+        },
+        # 这些是固定青色，模拟 loguru 的 <cyan> 标签
+        "name":      {k: "cyan" for k in log_colors},
+        "funcName":  {k: "cyan" for k in log_colors},
+        "lineno":    {k: "cyan" for k in log_colors},
+        "asctime":   {k: "green" for k in log_colors},  # 时间绿色
+    }
+
+
+    return colorlog.ColoredFormatter(
+        fmt=(
+            "%(asctime_log_color)s%(asctime)s.%(msecs)03d%(reset)s"
+            " | %(levelname_log_color)s%(levelname)-8s%(reset)s"
+            " | %(name_log_color)s%(name)s%(reset)s"
+            ":%(funcName_log_color)s%(filename)s%(reset)s"
+            ":%(funcName_log_color)s%(funcName)s%(reset)s"
+            ":%(lineno_log_color)s%(lineno)d%(reset)s"
+            " - %(message_log_color)s%(message)s%(reset)s"
+        ),
+        datefmt="%Y-%m-%d %H:%M:%S",
+        log_colors=log_colors,
+        secondary_log_colors=secondary,
+        style="%",
+    )
+
+# ---------- 4) 获取 logger（贴近 loguru 的 add 行为） ----------
+def get_logger(level: str = None) -> logging.Logger:
+    """返回一个名为 'DataFlow' 的 logger：
+    - 控制台输出分流：<ERROR 到 stdout；>=ERROR 到 stderr
+    - 颜色与格式尽量贴近 loguru 默认
+    - 避免重复添加 handler
+    """
+    if level is None:
+        level = os.getenv("DF_LOGGING_LEVEL", "INFO")
+
+    logger = logging.getLogger("DataFlow")
+    logger.setLevel(level)
+    logger.propagate = False  # 避免向 root 传播造成重复输出
+
+    if logger.handlers:
+        # 已初始化过，直接返回（可根据需要改成更新 handler 的级别/格式）
+        return logger
+
+    colored_fmt = _make_colored_formatter()
+
+    # stdout：DEBUG/INFO/SUCCESS/WARNING
+    h_out = logging.StreamHandler(stream=sys.stdout)
+    h_out.setLevel(level)
+    h_out.addFilter(MaxLevelFilter(logging.ERROR))
+    h_out.setFormatter(colored_fmt)
+
+    # stderr：ERROR/CRITICAL
+    h_err = logging.StreamHandler(stream=sys.stderr)
+    h_err.setLevel(level)
+    h_err.addFilter(MinLevelFilter(logging.ERROR))
+    h_err.setFormatter(colored_fmt)
+
+    logger.addHandler(h_out)
+    logger.addHandler(h_err)
+    return logger

dataflow/operators/__init__.py

+# from .eval import *
+# from .generate import *
+# from .filter import *
+# from .refine import *

dataflow/operators/agentic_rag/__init__.py

+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    # filter
+
+    # eval
+    from .eval.agenticrag_qaf1_sample_evaluator import AgenticRAGQAF1SampleEvaluator
+    # generate
+    from .generate.agenticrag_atomic_task_generator import AgenticRAGAtomicTaskGenerator
+    from .generate.agenticrag_depth_qa_generator import AgenticRAGDepthQAGenerator
+    from .generate.agenticrag_width_qa_generator import AgenticRAGWidthQAGenerator
+
+else:
+    import sys
+    from dataflow.utils.registry import LazyLoader, generate_import_structure_from_type_checking
+
+
+
+    cur_path = "dataflow/operators/agentic_rag/"
+
+
+    _import_structure = generate_import_structure_from_type_checking(__file__, cur_path)
+    sys.modules[__name__] = LazyLoader(__name__, "dataflow/operators/agentic_rag/", _import_structure)

dataflow/operators/agentic_rag/eval/agenticrag_qaf1_sample_evaluator.py

+import re
+import string
+from collections import Counter
+from tqdm import tqdm
+import pandas as pd
+from dataflow.core import OperatorABC
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow import get_logger
+
+
+@OPERATOR_REGISTRY.register()
+class AgenticRAGQAF1SampleEvaluator(OperatorABC):
+
+    def __init__(self):
+        self.logger = get_logger()
+        self.logger.info(f"Initializing {self.__class__.__name__}...")
+        self.logger.info(f"{self.__class__.__name__} initialized.")
+
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        return "用于评估预测答案与多个参考答案之间的 F1 分数"
+
+    def _validate_dataframe(self, dataframe: pd.DataFrame):
+        required_keys = [self.prediction_key, self.ground_truth_key]
+        forbidden_keys = [self.output_key ]
+
+        missing = [k for k in required_keys if k not in dataframe.columns]
+        conflict = [k for k in forbidden_keys if k in dataframe.columns]
+
+        if missing:
+            raise ValueError(f"Missing required column(s): {missing}")
+        if conflict:
+            raise ValueError(f"The following column(s) already exist and would be overwritten: {conflict}")
+
+    def normalize_answer(self, s: str) -> str:
+        def remove_articles(text):
+            return re.sub(r"\b(a|an|the)\b", " ", text)
+
+        def white_space_fix(text):
+            return " ".join(text.split())
+
+        def remove_punc(text):
+            exclude = set(string.punctuation)
+            return "".join(ch for ch in text if ch not in exclude)
+
+        def lower(text):
+            return text.lower()
+
+        return white_space_fix(remove_articles(remove_punc(lower(s))))
+
+    def compute_f1(self, prediction: str, ground_truths) -> float:
+        if prediction is None or ground_truths is None:
+            return 0.0
+
+        if isinstance(ground_truths, str):
+            ground_truths = [ground_truths]
+
+        max_f1 = 0.0
+
+        for ground_truth in ground_truths:
+            if ground_truth is None:
+                continue
+
+            normalized_prediction = self.normalize_answer(prediction)
+            normalized_ground_truth = self.normalize_answer(ground_truth)
+
+            if normalized_prediction in ["yes", "no", "noanswer"] or normalized_ground_truth in ["yes", "no", "noanswer"]:
+                if normalized_prediction != normalized_ground_truth:
+                    continue
+
+            pred_tokens = normalized_prediction.split()
+            gold_tokens = normalized_ground_truth.split()
+            common = Counter(pred_tokens) & Counter(gold_tokens)
+            num_same = sum(common.values())
+
+            if num_same == 0:
+                continue
+
+            precision = num_same / len(pred_tokens)
+            recall = num_same / len(gold_tokens)
+            f1 = (2 * precision * recall) / (precision + recall)
+            max_f1 = max(max_f1, f1)
+
+        return max_f1
+
+    def eval(self, dataframe: pd.DataFrame) -> list:
+        self.logger.info(f"Evaluating {self.output_key}...")
+        f1_scores = []
+
+        for _, row in tqdm(dataframe.iterrows(), total=len(dataframe), desc="F1Scorer Evaluating..."):
+            prediction = row.get(self.prediction_key, None)
+            ground_truths = row.get(self.ground_truth_key, None)
+            score = self.compute_f1(prediction, ground_truths)
+            f1_scores.append(score)
+
+        self.logger.info("Evaluation complete!")
+        return f1_scores
+
+    def run(self, 
+            storage: DataFlowStorage, 
+            input_prediction_key:str ="refined_answer",
+            input_ground_truth_key:str ="golden_doc_answer",
+            output_key:str ="F1Score",
+            ):
+        dataframe = storage.read("dataframe")
+        self.output_key = output_key
+        self.prediction_key = input_prediction_key
+        self.ground_truth_key = input_ground_truth_key
+        self._validate_dataframe(dataframe)
+        scores = self.eval(dataframe)
+        dataframe[self.output_key] = scores
+        storage.write(dataframe)

dataflow/operators/agentic_rag/generate/agenticrag_atomic_task_generator.py

+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow  import get_logger
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+from dataflow.core import LLMServingABC
+
+from dataflow.prompts.agenticrag import (
+AtomicTaskGeneratorGetIdentifierPrompt, 
+AtomicTaskGeneratorGetConlcusionPrompt, 
+AtomicTaskGeneratorQuestionPrompt, 
+AtomicTaskGeneratorCleanQAPrompt, 
+AtomicTaskGeneratorAnswerPrompt, 
+AtomicTaskGeneratorRecallScorePrompt, 
+AtomicTaskGeneratorOptionalAnswerPrompt, 
+AtomicTaskGeneratorGoldenDocAnswerPrompt
+)
+from dataflow.core.prompt import prompt_restrict
+
+import pandas as pd
+import json
+import string
+import re
+from collections import Counter
+
+@prompt_restrict(
+    AtomicTaskGeneratorGetIdentifierPrompt, 
+    AtomicTaskGeneratorGetConlcusionPrompt, 
+    AtomicTaskGeneratorQuestionPrompt, 
+    AtomicTaskGeneratorCleanQAPrompt, 
+    AtomicTaskGeneratorAnswerPrompt, 
+    AtomicTaskGeneratorRecallScorePrompt, 
+    AtomicTaskGeneratorOptionalAnswerPrompt, 
+    AtomicTaskGeneratorGoldenDocAnswerPrompt
+)
+@OPERATOR_REGISTRY.register()
+class AgenticRAGAtomicTaskGenerator(OperatorABC):
+    def __init__(self,
+                 llm_serving: LLMServingABC = None,
+                 data_num : int = 100,
+                 max_per_task: int = 10, # Limit the number of candidate tasks per input
+                 max_question: int = 10,  # Limit the question of each doc
+                #  prompt_template = None # prompt is fix
+                 ):
+        self.logger= get_logger()
+        self.llm_serving = llm_serving
+        self.data_num, self.max_per_task, self.max_question = data_num, max_per_task, max_question
+
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        if lang == "zh":
+            return (
+                "该算子用于为提供的文本内容生成合适的高质量问题与可验证答案。\n\n"
+                "输入参数：\n"
+                "- input_key: 输入文本内容字段名（默认值：\"prompts\"）\n"
+                "- output_question_key: 输出问题字段名（默认值：\"question\"）\n"
+                "- output_answer_key: 输出答案字段名（默认值：\"answer\"）\n"
+                "- output_refined_answer_key: 输出精炼答案字段名（默认值：\"refined_answer\"）\n"
+                "- output_optional_answer_key: 输出可替代精炼答案字段名（默认值：\"optional_answer\"）\n"
+                "- output_golden_doc_answer_key: 输出黄金文档回答字段名（默认值：\"golden_doc_answer\"）\n"
+            )
+        elif lang == "en":
+            return (
+                "This operator is used to generate appropriate high-quality questions and verifiable answers for the provided text content."
+                "Input Parameters:\n"
+                "- input_key: Field name of the input text content (default: \"prompts\")\n"
+                "- output_question_key: Field name for the output question (default: \"question\")\n"
+                "- output_answer_key: Field name for the output answer (default: \"answer\")\n"
+                "- output_refined_answer_key: Field name for the output refined answer (default: \"refined_answer\")\n"
+                "- output_optional_answer_key: Field name for the output optional refined answer (default: \"optional_answer\")\n"
+                "- output_golden_doc_answer_key: Field name for the output answer based on gold documents (default: \"golden_doc_answer\")\n"
+            )
+        else:
+            return "AtomicTaskGenerator generate appropriate high-quality questions and verifiable answers for the provided text content."
+    
+    def _validate_dataframe(self, dataframe: pd.DataFrame):
+        required_keys = [self.input_key]
+        forbidden_keys = [self.output_refined_answer_key, self.output_answer_key, self.output_question_key, self.output_optional_answer_key, self.output_llm_answer_key, self.output_golden_doc_answer_key]
+
+        missing = [k for k in required_keys if k not in dataframe.columns]
+        conflict = [k for k in forbidden_keys if k in dataframe.columns]
+
+        if missing:
+            raise ValueError(f"Missing required column(s): {missing}")
+        if conflict:
+            raise ValueError(f"The following column(s) already exist and would be overwritten: {conflict}")
+
+    def _reformat_prompt(self, dataframe, prompt_type: str = None):
+        """
+        Reformat the prompts in the dataframe to generate LLM input.
+        All input columns are expected to be strings.
+        """
+        if prompt_type == "get_identifier":
+            self.prompt_template = AtomicTaskGeneratorGetIdentifierPrompt()
+            input_prompts = dataframe[self.input_key].tolist()
+            system_prompt = self.prompt_template.build_system_prompt()
+            prompts = [self.prompt_template.build_prompt(p) for p in input_prompts]
+
+        elif prompt_type == "get_conclusion":
+            self.prompt_template = AtomicTaskGeneratorGetConlcusionPrompt()
+            input_prompts = dataframe[self.input_key].tolist()
+            system_prompt = self.prompt_template.build_system_prompt()
+            prompts = [self.prompt_template.build_prompt(p) for p in input_prompts]
+
+        elif prompt_type == "init_question":
+            self.prompt_template = AtomicTaskGeneratorQuestionPrompt()
+            candidate_strs = dataframe["candidate_tasks_str"].tolist()
+            raw_identifiers = dataframe["identifier"].tolist()
+            system_prompt = self.prompt_template.build_system_prompt()
+            prompts = []
+            for s, raw_id in zip(candidate_strs, raw_identifiers):
+                try:
+                    # 解析 candidate_tasks_str 字段
+                    task_item = json.loads(self._clean_json_block(s))
+
+                    # 清理并解析 identifier 字段
+                    clean_id_str = self._clean_json_block(raw_id)
+                    identifier_obj = json.loads(clean_id_str)
+                    identifier = identifier_obj.get("content_identifier", "Unknown")
+
+                    prompts.append(
+                        self.prompt_template.build_prompt(identifier, task_item["conclusion"], task_item["R"])
+                    )
+                except Exception as e:
+                    print(f"[WARN] Failed to parse candidate_tasks_str or identifier: {e} | value: {s} | id: {raw_id}")
+                    prompts.append("")  # fallback
+
+        elif prompt_type == "clean_qa":
+            self.prompt_template = AtomicTaskGeneratorCleanQAPrompt()
+            questions = dataframe[self.output_question_key].tolist()
+            answers = dataframe[self.output_answer_key].tolist()
+            system_prompt = self.prompt_template.build_system_prompt()
+            prompts = [
+                self.prompt_template.build_prompt({"question": q, "original_answer": a})
+                for q, a in zip(questions, answers)
+            ]
+        elif prompt_type == "llm_answer":
+            self.prompt_template = AtomicTaskGeneratorAnswerPrompt()
+            questions = dataframe[self.output_question_key].tolist()
+            system_prompt = ""
+            prompts = [
+                self.prompt_template.build_prompt(question) for question in questions
+            ]
+        elif prompt_type == "get_recall_score":
+            self.prompt_template = AtomicTaskGeneratorRecallScorePrompt()
+            golden_answers = dataframe[self.output_refined_answer_key].tolist()
+            llm_answers = dataframe[self.output_llm_answer_key]
+            system_prompt = self.prompt_template.build_system_prompt()
+            prompts = [
+                self.prompt_template.build_prompt(golden_answer, llm_answer) for golden_answer, llm_answer in zip(golden_answers, llm_answers)
+            ]
+        elif prompt_type == "get_golden_answer_score":
+            self.prompt_template = AtomicTaskGeneratorRecallScorePrompt()
+            golden_answers = dataframe[self.output_refined_answer_key].tolist()
+            llm_answers = dataframe[self.output_golden_doc_answer_key]
+            system_prompt = self.prompt_template.build_system_prompt()
+            prompts = [
+                self.prompt_template.build_prompt(golden_answer, llm_answer) for golden_answer, llm_answer in zip(golden_answers, llm_answers)
+            ]
+        elif prompt_type == "more_optional_answer":
+            self.prompt_template = AtomicTaskGeneratorOptionalAnswerPrompt()
+            answers = dataframe[self.output_refined_answer_key].tolist()
+            system_prompt = self.prompt_template.build_system_prompt()
+            prompts = [
+                self.prompt_template.build_prompt(answer) for answer in answers
+            ]
+        elif prompt_type == "golden_doc_answer":
+            self.prompt_template = AtomicTaskGeneratorGoldenDocAnswerPrompt()
+            golden_docs = dataframe[self.input_key].tolist()
+            questions = dataframe[self.output_question_key].tolist()
+            system_prompt = ""
+            prompts = [
+                self.prompt_template.build_prompt(golden_doc, question) 
+                for golden_doc, question in zip(golden_docs, questions)
+            ]
+        else:
+            raise ValueError(f"Unknown prompt_type: {prompt_type}")
+
+        return system_prompt, prompts
+
+    
+    def _clean_json_block(self, item: str) -> str:
+        return item.strip().removeprefix("```json").removeprefix("```").removesuffix("```").strip()
+
+    def recall_score(self, dataframe):
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "get_recall_score")
+        recall_scores = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+        valid_scores = []
+        for score_str in recall_scores:
+            try:
+                score_dict = json.loads(self._clean_json_block(score_str))
+                valid_scores.append(score_dict["answer_score"])
+            except Exception as e:
+                print("recall score_str error:", score_str, "\nError:", e)
+                valid_scores.append(0)
+                continue
+        return valid_scores
+
+    def recall_score_golden_doc(self, dataframe):
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "get_golden_answer_score")
+        recall_scores = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+        valid_scores = []
+        for score_str in recall_scores:
+            try:
+                score_dict = json.loads(self._clean_json_block(score_str))
+                valid_scores.append(score_dict["answer_score"])
+            except Exception as e:
+                print("recall score_str error:", score_str, "\nError:", e)
+                valid_scores.append(0)
+                continue
+        return valid_scores
+
+    def more_optional_answer(self, dataframe):
+        original_answer = dataframe[self.output_refined_answer_key]
+        system_prompt, user_prompts = self._reformat_prompt(dataframe, "more_optional_answer")
+        optional_answers = self.llm_serving.generate_from_input(user_prompts, system_prompt)
+        valid_answers = []
+        for optional_answer in optional_answers:
+            try:
+                if isinstance(optional_answer, str):
+                    optional_answer = json.loads(self._clean_json_block(optional_answer))
+                    valid_answers.append(optional_answer)
+                else:
+                    valid_answers.append(optional_answer)
+            except Exception as e:
+                print(f"Error parsing optional answer: {optional_answer} | Error: {e}")
+                valid_answers.append(original_answer)
+        return valid_answers
+        
+    def normalize_answer(self, s):
+        def remove_articles(text):
+            return re.sub(r"\b(a|an|the)\b", " ", text)
+
+        def white_space_fix(text):
+            return " ".join(text.split())
+
+        def remove_punc(text):
+            exclude = set(string.punctuation)
+            return "".join(ch for ch in text if ch not in exclude)
+
+        def lower(text):
+            return text.lower()
+
+        return white_space_fix(remove_articles(remove_punc(lower(s))))
+
+    def get_f1_score(self, dataframe):
+
+        f1_scores = []
+        for idx, row in dataframe.iterrows():
+            prediction = row[self.output_golden_doc_answer_key]
+            ground_truths = row[self.output_optional_answer_key]
+
+            final_metric = {"f1": 0, "precision": 0, "recall": 0}
+
+            if ground_truths is None or prediction is None:
+                f1_scores.append(final_metric['f1'])
+                continue
+                
+            if isinstance(ground_truths, str):
+                ground_truths = [ground_truths]
+
+            for ground_truth in ground_truths:
+                
+                if ground_truth is None:
+                    continue
+
+                normalized_prediction = self.normalize_answer(prediction)
+                normalized_ground_truth = self.normalize_answer(ground_truth)
+
+                if normalized_prediction in ["yes", "no", "noanswer"] and normalized_prediction != normalized_ground_truth:
+                    continue
+
+                if normalized_ground_truth in ["yes", "no", "noanswer"] and normalized_prediction != normalized_ground_truth:
+                    continue
+
+                prediction_tokens = normalized_prediction.split()
+                ground_truth_tokens = normalized_ground_truth.split()
+                common = Counter(prediction_tokens) & Counter(ground_truth_tokens)
+                num_same = sum(common.values())
+                if num_same == 0:
+                    continue
+
+                precision = 1.0 * num_same / len(prediction_tokens)
+                recall = 1.0 * num_same / len(ground_truth_tokens)
+                f1 = (2 * precision * recall) / (precision + recall)
+
+                final_metric["precision"] = max(precision, final_metric["precision"])
+                final_metric["recall"] = max(recall, final_metric["recall"])
+                final_metric["f1"] = max(f1, final_metric["f1"])
+
+            f1_scores.append(final_metric['f1'])
+
+        return f1_scores
+        
+        
+    def run(
+        self,
+        storage: DataFlowStorage,
+        input_key: str = "prompts",
+        output_question_key: str = "question",
+        output_answer_key:str = "answer",
+        output_refined_answer_key:str = "refined_answer",
+        output_optional_answer_key: str = "optional_answer",
+        output_llm_answer_key: str = "llm_answer",
+        output_golden_doc_answer_key: str = "golden_doc_answer",
+    ):
+        self.input_key, self.output_question_key = input_key, output_question_key
+
+        self.output_answer_key, self.output_refined_answer_key, self.output_optional_answer_key = output_answer_key, output_refined_answer_key, output_optional_answer_key
+
+        self.output_llm_answer_key, self.output_golden_doc_answer_key = output_llm_answer_key, output_golden_doc_answer_key
+
+
+        dataframe = storage.read("dataframe").iloc[:]
+        self._validate_dataframe(dataframe)
+
+        # === Step 0: Get identifier
+        self.logger.info("Get identifier...")
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "get_identifier")
+        identifiers = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+
+        # === Step 1: Get conclusions
+        self.logger.info("Get conclusions...")
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "get_conclusion")
+        conclusions = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+
+        # === Expand each conclusion into multiple candidate tasks (rows)
+        expanded_rows = []
+        # Use to_dict('records') instead of itertuples() to preserve column names with special characters (e.g., "user:contents")
+        for idx, (row, output_str, identifier) in enumerate(zip(dataframe.to_dict('records'), conclusions, identifiers)):
+            try:
+                parsed = json.loads(self._clean_json_block(output_str))
+                parsed = parsed[:self.max_per_task] if isinstance(parsed, list) else parsed
+            except Exception as e:
+                print(f"[WARN] JSON parse failed at idx={idx}: {e} | output: {output_str}")
+                continue
+
+            if not isinstance(parsed, list):
+                continue
+
+            for item in parsed:
+                if isinstance(item, dict) and "conclusion" in item and "R" in item:
+                    expanded_rows.append({
+                        **row,  # row is already a dict now
+                        "identifier": str(identifier),
+                        "candidate_tasks_str": json.dumps(item, ensure_ascii=False)
+                    })
+
+        if not expanded_rows:
+            self.logger.warning("No valid candidate tasks extracted.")
+            return
+
+        dataframe = pd.DataFrame(expanded_rows)
+
+        # === Step 2: Generate questions based on conclusion + reasoning
+        self.logger.info("Generate questions based on conclusion + reasoning...")
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "init_question")
+        question_outputs = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+
+        questions = []
+        answers = []
+        valid_rows = []
+
+        # Use to_dict('records') instead of itertuples() to preserve column names with special characters
+        for idx, (res, row) in enumerate(zip(question_outputs, dataframe.to_dict('records'))):
+            try:
+                parsed = json.loads(self._clean_json_block(res))
+            except Exception as e:
+                print(f"[WARN] Failed to parse question JSON at idx={idx}: {e} | res: {res}")
+                continue
+
+            if isinstance(parsed, dict) and "Q" in parsed:
+                question = parsed["Q"]
+                try:
+                    task = json.loads(self._clean_json_block(row['candidate_tasks_str']))
+                    answer = task.get("conclusion", "")
+                except Exception:
+                    answer = ""
+                valid_rows.append(row)  # row is already a dict
+                questions.append(str(question))
+                answers.append(str(answer))
+
+        if not valid_rows:
+            self.logger.warning("No valid QA pairs generated.")
+            return
+
+        dataframe = pd.DataFrame(valid_rows)
+        dataframe[self.output_question_key] = questions
+        dataframe[self.output_answer_key] = answers
+
+        # === Step 3: Clean QA
+        self.logger.info("Clean QA...")
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "clean_qa")
+        clean_outputs = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+
+        final_answers = []
+
+        for idx, res in enumerate(clean_outputs):
+            try:
+                parsed = json.loads(self._clean_json_block(res))
+                final_answers.append(str(parsed.get("refined_answer", "")))
+            except Exception as e:
+                print(f"[WARN] Failed to parse cleaned QA at idx={idx}: {e} | res: {res}")
+                final_answers.append("")
+
+        dataframe[self.output_refined_answer_key] = final_answers
+
+        # Verify module
+        self.logger.info("LLM reasoning verify...")
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "llm_answer")
+        llm_answer_results = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+        dataframe[self.output_llm_answer_key] = llm_answer_results
+        
+        llm_score = self.recall_score(dataframe)
+        dataframe["llm_score"] = llm_score
+        dataframe = dataframe[dataframe["llm_score"] < 1].reset_index(drop=True)
+
+        if dataframe.empty:
+            self.logger.warning("No data left after LLM score filtering. All questions were answered correctly by LLM.")
+            return
+
+        self.logger.info("Get golden doc answer...")
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "golden_doc_answer")
+        llm_answer_results = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+        dataframe[self.output_golden_doc_answer_key] = llm_answer_results
+
+
+        # golden doc answer verify
+        self.logger.info("Golden doc LLM verifying...")
+        golden_doc_score = self.recall_score_golden_doc(dataframe)
+        dataframe["golden_doc_score"] = golden_doc_score
+        dataframe = dataframe[dataframe["golden_doc_score"] >= 1].reset_index(drop=True)
+
+        # more optional answer
+        self.logger.info("Generating more optional answer...")
+        dataframe[self.output_optional_answer_key] = self.more_optional_answer(dataframe)
+        
+        dataframe = (
+            dataframe.groupby(input_key, group_keys=False)
+            .apply(lambda x: x.head(self.max_question))
+            .reset_index(drop=True)
+        )
+
+        output_file = storage.write(dataframe)
+        self.logger.info(f"Results saved to {output_file}")
+        return "identifier", "candidate_tasks_str", "llm_score", "golden_doc_score", self.output_question_key, self.output_answer_key, self.output_refined_answer_key, self.output_optional_answer_key, self.output_llm_answer_key, self.output_golden_doc_answer_key
\ No newline at end of file

dataflow/operators/agentic_rag/generate/agenticrag_depth_qa_generator.py

+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow  import get_logger
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+from dataflow.core import LLMServingABC
+from dataflow.core.prompt import prompt_restrict
+
+from dataflow.prompts.agenticrag import (
+    DepthQAGeneratorGetIdentifierPrompt,
+    DepthQAGeneratorBackwardTaskPrompt,
+    DepthQAGeneratorSupersetCheckPrompt,
+    DepthQAGeneratorQuestionPrompt,
+    DepthQAGeneratorAnswerPrompt,
+    DepthQAGeneratorRecallScorePrompt
+)
+import pandas as pd
+import json
+
+@prompt_restrict(
+    DepthQAGeneratorGetIdentifierPrompt,
+    DepthQAGeneratorBackwardTaskPrompt,
+    DepthQAGeneratorSupersetCheckPrompt,
+    DepthQAGeneratorQuestionPrompt,
+    DepthQAGeneratorAnswerPrompt,
+    DepthQAGeneratorRecallScorePrompt
+)
+@OPERATOR_REGISTRY.register()
+class AgenticRAGDepthQAGenerator(OperatorABC):
+    def __init__(self,
+                 llm_serving: LLMServingABC = None,
+                 n_rounds:int = 2,
+                #  prompt_template = None # prompt is fix
+                 ):
+        self.logger= get_logger()
+        self.llm_serving = llm_serving
+        self.n_rounds = n_rounds
+
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        if lang == "zh":
+            return (
+                "该算子以已有问答生成更深度的问题。\n\n"
+                "输入参数：\n"
+                "- input_key: 输入字段名（默认值：\"question\"）\n"
+                "- output_key: 输出字段名（默认值：\"depth_question\"）\n"
+            )
+        elif lang == "en":
+            return (
+                "This operator is used to generate deeper questions based on existing QA pairs."
+                "Input Parameters:\n"
+                "- input_key: Field name for the input (default: \"question\")\n"
+                "- output_key: Field name for the output (default: \"depth_question\")\n"
+            )
+        else:
+            return "DepthQAGenerator generate deeper questions based on existing QA pairs."
+    
+    def _validate_dataframe(self, dataframe: pd.DataFrame):
+        required_keys = [self.input_key]
+        forbidden_keys = [self.output_key]
+
+        missing = [k for k in required_keys if k not in dataframe.columns]
+        conflict = [k for k in forbidden_keys if k in dataframe.columns]
+
+        if missing:
+            raise ValueError(f"Missing required column(s): {missing}")
+        if conflict:
+            raise ValueError(f"The following column(s) already exist and would be overwritten: {conflict}")
+
+    def _reformat_prompt(self, dataframe, prompt_type:str = None):
+        """
+        Reformat the prompts in the dataframe to generate questions.
+        """
+        if prompt_type == "get_identifier":
+            self.prompt_template = DepthQAGeneratorGetIdentifierPrompt()
+            input_prompts = dataframe[self.input_key].tolist()
+            system_prompts = self.prompt_template.build_system_prompt()
+            prompts = [self.prompt_template.build_prompt(input_prompts) for input_prompts in input_prompts]
+        elif prompt_type == "get_backward":
+            self.prompt_template = DepthQAGeneratorBackwardTaskPrompt()
+            input_prompts = dataframe[self.identifier_key].tolist()
+            system_prompts = ""
+            prompts = [self.prompt_template.build_prompt(input_prompts) for input_prompts in input_prompts]
+        elif prompt_type == "check_superset":
+            self.prompt_template = DepthQAGeneratorSupersetCheckPrompt()
+            new_identifiers = dataframe[self.new_identifier_key].tolist()
+            relations = dataframe[self.relation_key].tolist()
+            identifiers = dataframe[self.identifier_key].tolist()
+            system_prompts = self.prompt_template.build_system_prompt()
+            prompts = [self.prompt_template.build_prompt(new_id, relation, identifier) for new_id, relation, identifier in zip(new_identifiers, relations, identifiers)]
+        elif prompt_type == "get_new_question":
+            self.prompt_template = DepthQAGeneratorQuestionPrompt()
+            new_identifiers = dataframe[self.new_identifier_key].tolist()
+            relations = dataframe[self.relation_key].tolist()
+            identifiers = dataframe[self.identifier_key].tolist()
+            system_prompts = self.prompt_template.build_system_prompt()
+            prompts = [self.prompt_template.build_prompt(new_id, relation, identifier) for new_id, relation, identifier in zip(new_identifiers, relations, identifiers)]
+        elif prompt_type == "llm_answer":
+            self.prompt_template = DepthQAGeneratorAnswerPrompt()
+            questions = dataframe[self.input_key].tolist()
+            system_prompts = ""
+            prompts = [
+                self.prompt_template.build_prompt(question) for question in questions
+            ]
+        elif prompt_type == "get_recall_score":
+            self.prompt_template = DepthQAGeneratorRecallScorePrompt()
+            golden_answers = dataframe["refined_answer"].tolist()
+            llm_answers = dataframe["llm_answer"]
+            system_prompts = self.prompt_template.build_system_prompt()
+            prompts = [
+                self.prompt_template.build_prompt(golden_answer, llm_answer) for golden_answer, llm_answer in zip(golden_answers, llm_answers)
+            ]
+        else:
+            raise ValueError(f"Unknown prompt_type: {prompt_type}")
+        return system_prompts, prompts
+    
+    def _clean_json_block(self, item: str) -> str:
+        return item.strip().removeprefix("```json").removeprefix("```").removesuffix("```").strip()
+
+    def recall_score(self, dataframe):
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "get_recall_score")
+        recall_scores = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+        valid_scores = []
+        for score_str in recall_scores:
+            if score_str is not None:
+                try:
+                    score_dict = json.loads(score_str)
+                    valid_scores.append(score_dict["answer_score"])
+                except (json.JSONDecodeError, KeyError):
+                    print(score_str)
+                    valid_scores.append(0)
+                    continue
+        return valid_scores
+
+    def run(
+            self,
+            storage: DataFlowStorage,
+            input_key:str = "question",
+            output_key:str = "depth_question",
+            ):
+        self.input_key, self.output_key = input_key, output_key
+        dataframe = storage.read("dataframe")
+        self._validate_dataframe(dataframe)
+
+        if "identifier" not in dataframe.columns:
+            sys_prompts, user_prompts = self._reformat_prompt(dataframe, "get_identifier")
+            identifiers = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+            dataframe["identifier"] = identifiers
+
+        # step0: get identifier
+        for round_id in range(1, self.n_rounds + 1):
+            self.logger.info(f"=== Iteration Round {round_id} ===")
+
+            # Use identifier from previous round
+            self.identifier_key = "identifier" if round_id == 1 else f"new_identifier_{round_id - 1}"
+            self.new_identifier_key, self.relation_key =f"new_identifier_{round_id}", f"relation_{round_id}"
+            # Backward Step:
+            # step1: Generate relation and superset
+            sys_prompts, user_prompts = self._reformat_prompt(dataframe, "get_backward")
+            backward_results = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+            
+            identifiers = []
+            relations = []
+            valid_indices = []
+
+            for idx, result in enumerate(backward_results):
+                try:
+                    if isinstance(result, str):
+                        result = json.loads(self._clean_json_block(result))
+
+                    if isinstance(result, dict) and "identifier" in result and "relation" in result:
+                        identifiers.append(result["identifier"])
+                        relations.append(result["relation"])
+                        valid_indices.append(idx)
+                    else:
+                        self.logger.warning(f"[Skipped]: Result at index {idx} is invalid: {result}")
+                except Exception as e:
+                    self.logger.warning(f"[Error]: Failed to parse backward result at index {idx}: {e}")
+                    continue
+
+            dataframe = dataframe.iloc[valid_indices].copy()
+            dataframe[self.new_identifier_key] = identifiers
+            dataframe[self.relation_key] = relations
+
+            # step2: Check if superset is valid
+            sys_prompts, user_prompts = self._reformat_prompt(dataframe, "check_superset")
+            check_results = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+            
+            valid_indices = []
+            for idx, result in enumerate(check_results):
+                try:
+                    if isinstance(result, str):
+                        result = json.loads(self._clean_json_block(result))
+
+                    if isinstance(result, dict) and "new_query" in result:
+                        if result["new_query"] == "valid":
+                            valid_indices.append(idx)
+                    else:
+                        self.logger.warning(f"[Skipped]: Result at index {idx} is invalid: {result}")
+                except Exception as e:
+                    self.logger.warning(f"[Error]: Failed to check superset result at index {idx}: {e}")
+                    continue
+
+            dataframe = dataframe.iloc[valid_indices].copy()
+
+            # step3: Generate question based on superset and relation
+            sys_prompts, user_prompts = self._reformat_prompt(dataframe, "get_new_question")
+            check_results = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+
+            new_queries = []
+            valid_indices = []
+            for idx, result in enumerate(check_results):
+                try:
+                    if isinstance(result, str):
+                        result = json.loads(self._clean_json_block(result))
+
+                    if isinstance(result, dict):
+                        new_queries.append(result["new_query"])
+                        valid_indices.append(idx)
+                    else:
+                        self.logger.warning(f"[Skipped]: Result at index {idx} is invalid: {result}")
+                except Exception as e:
+                    self.logger.warning(f"[Error]: Failed to check superset result at index {idx}: {e}")
+                    continue
+
+            dataframe = dataframe.iloc[valid_indices].copy()
+            question_key = f"{output_key}_{round_id}"
+            dataframe[question_key] = new_queries
+            
+            # Verify module
+            sys_prompts, user_prompts = self._reformat_prompt(dataframe, "llm_answer")
+            llm_answer_results = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+
+            dataframe["llm_answer"] = llm_answer_results
+            llm_score = self.recall_score(dataframe)
+            dataframe["llm_score"] = llm_score
+            dataframe = dataframe[dataframe["llm_score"] < 1].drop(columns=["llm_score"]).reset_index(drop=True)
+            dataframe = dataframe.drop(columns="llm_answer")
+
+        output_file = storage.write(dataframe)
+        self.logger.info(f"Results saved to {output_file}")
+        
+        return [f"new_identifier_{i}" for i in range(1, self.n_rounds + 1)] + \
+               [f"relation_{i}" for i in range(1, self.n_rounds + 1)] + \
+               [f"{output_key}_{i}" for i in range(1, self.n_rounds + 1)]
\ No newline at end of file

dataflow/operators/agentic_rag/generate/agenticrag_width_qa_generator.py

+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow  import get_logger
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+from dataflow.core import LLMServingABC
+from dataflow.core.prompt import prompt_restrict
+
+from dataflow.prompts.agenticrag import (
+    WidthQAGeneratorMergePrompt,
+    WidthQAGeneratorOriginCheckPrompt,
+    WidthQAGeneratorQuestionVerifyPrompt,
+    WidthQAGeneratorAnswerPrompt,
+    WidthQAGeneratorRecallScorePrompt
+)
+import pandas as pd
+import json
+
+@prompt_restrict(
+    WidthQAGeneratorMergePrompt,
+    WidthQAGeneratorOriginCheckPrompt,
+    WidthQAGeneratorQuestionVerifyPrompt,
+    WidthQAGeneratorAnswerPrompt,
+    WidthQAGeneratorRecallScorePrompt
+)
+@OPERATOR_REGISTRY.register()
+class AgenticRAGWidthQAGenerator(OperatorABC):
+    def __init__(self,
+                 llm_serving: LLMServingABC = None,
+                #  prompt_template = None  # prompt is fix
+                 ):
+        self.logger= get_logger()
+        self.llm_serving = llm_serving
+
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        if lang == "zh":
+            return (
+                "该算子用于结合两个问答，生成新的问题。\n\n"
+                "输入参数：\n"
+                "- input_question_key: 输入问题字段名（默认值：\"question\"）\n"
+                "- input_identifier_key: 输入标识符字段名（默认值：\"identifier\"）\n"
+                "- input_answer_key: 输入答案字段名（默认值：\"answer\"）\n"
+                "- output_question_key: 输出问题字段名（默认值：\"generated_width_task\"）\n"
+            )
+        elif lang == "en":
+            return (
+                "This operator is used to combine two QA pairs and generate a new question."
+                "Input Parameters:\n"
+                "- input_question_key: Field name for the input question (default: \"question\")\n"
+                "- input_identifier_key: Field name for the input identifier (default: \"identifier\")\n"
+                "- input_answer_key: Field name for the input answer (default: \"answer\")\n"
+                "- output_question_key: Field name for the output question (default: \"generated_width_task\")\n"
+            )
+        else:
+            return "WidthQAGenerator combine two QA pairs and generate a new question."
+    
+    def _validate_dataframe(self, dataframe: pd.DataFrame):
+        required_keys = [self.input_question_key, self.input_answer_key, self.input_identifier_key]
+        forbidden_keys = [self.output_question_key, "content_identifier", "qa_index", "index", "original_answer", "original_question", "state"]
+
+        missing = [k for k in required_keys if k not in dataframe.columns]
+        conflict = [k for k in forbidden_keys if k in dataframe.columns]
+
+        if missing:
+            raise ValueError(f"Missing required column(s): {missing}")
+        if conflict:
+            raise ValueError(f"The following column(s) already exist and would be overwritten: {conflict}")
+
+    def _reformat_prompt(self, dataframe, prompt_type:str = None, input_batch: list[dict] = [None]):
+        """
+        Reformat the prompts in the dataframe to generate questions.
+        """
+        if prompt_type == "merge_prompt":
+            self.prompt_template = WidthQAGeneratorMergePrompt()
+            system_prompts = self.prompt_template.build_system_prompt()
+            prompts = [
+                self.prompt_template.build_prompt([input_batch[i], input_batch[i + 1]])
+                for i in range(len(input_batch) - 1)
+            ]
+        elif prompt_type == "check_origin":
+            self.prompt_template = WidthQAGeneratorOriginCheckPrompt()
+            input_batch = []
+            for idx, q, ori_q in zip(dataframe["index"], dataframe["question"], dataframe["original_question"]):
+                input_batch.append({
+                    "index": idx,
+                    "complex_question": q,
+                    "original_questions": ori_q if isinstance(ori_q, list) else [ori_q]
+                })
+            system_prompts = self.prompt_template.build_system_prompt()
+            prompts = [self.prompt_template.build_prompt(input) for input in input_batch]
+            return system_prompts, prompts
+        elif prompt_type == "question_verify":
+            self.prompt_template = WidthQAGeneratorQuestionVerifyPrompt()
+            input_batch = []
+            for idx, q in zip(dataframe["index"], dataframe[self.output_question_key]):
+                input_batch.append({
+                    "index": idx,
+                    "complex_question": q,
+                })
+            system_prompts = self.prompt_template.build_system_prompt()
+            prompts = [self.prompt_template.build_prompt(input) for input in input_batch]
+        elif prompt_type == "get_recall_score":
+            self.prompt_template = WidthQAGeneratorRecallScorePrompt()
+            golden_answers = dataframe["original_answer"].tolist()
+            llm_answers = dataframe["llm_answer"]
+            system_prompts = self.prompt_template.build_system_prompt()
+            prompts = [
+                self.prompt_template.build_prompt(golden_answer, llm_answer) for golden_answer, llm_answer in zip(golden_answers, llm_answers)
+            ]
+        else:
+            raise ValueError(f"Unknown prompt_type: {prompt_type}")
+        return system_prompts, prompts
+    
+    def _clean_json_block(self, item: str) -> str:
+        return item.strip().removeprefix("```json").removeprefix("```").removesuffix("```").strip()
+
+    def recall_score(self, dataframe):
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "get_recall_score")
+        recall_scores = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+        valid_scores = []
+        for score_str in recall_scores:
+            if score_str is not None:
+                try:
+                    score_dict = json.loads(score_str)
+                    valid_scores.append(score_dict["answer_score"])
+                except (json.JSONDecodeError, KeyError):
+                    print(score_str)
+                    valid_scores.append(0)
+                    continue
+        return valid_scores
+
+    def run(
+            self,
+            storage: DataFlowStorage,
+            input_question_key:str = "question",
+            input_identifier_key:str = "identifier",
+            input_answer_key:str = "answer",
+            output_question_key:str = "generated_width_task"
+            ):
+        self.input_question_key, self.input_identifier_key, self.input_answer_key, self.output_question_key = input_question_key, input_identifier_key,input_answer_key, output_question_key
+        dataframe = storage.read("dataframe")
+        self._validate_dataframe(dataframe)
+
+        # step0: merge_prompt
+        input_batch = []
+        for i, (_, row) in enumerate(dataframe.iterrows()):
+            input_batch.append({
+                "index": i,
+                "question": row[self.input_question_key],
+                "content_identifier": row[self.input_identifier_key],
+                "golden_answer": row[self.input_answer_key]
+            })
+
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "merge_prompt", input_batch)
+        merge_results = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+
+        merged_rows = []
+        for idx, result in enumerate(merge_results):
+            try:
+                if isinstance(result, str):
+                    result = json.loads(self._clean_json_block(result))[0]
+
+                if not isinstance(result, dict) or "question" not in result or "index" not in result:
+                    self.logger.warning(f"[Skipped]: Invalid result at index {idx}: {result}")
+                    continue
+
+                indices = result["index"] if isinstance(result["index"], list) else [result["index"]]
+                group_items = [input_batch[i] for i in indices]
+
+                merged_rows.append({
+                    "question": result["question"],
+                    "content_identifier": result["content_identifier"],
+                    "qa_index": indices,
+                    "index": idx,
+                    "original_answer": [item["golden_answer"] for item in group_items],
+                    "original_question": [item["question"] for item in group_items],
+                })
+
+            except Exception as e:
+                self.logger.warning(f"[Error]: Failed to parse merge result at index {idx}: {e}")
+                continue
+
+        dataframe = pd.DataFrame(merged_rows)
+        
+        # check queries
+        # Step 1: Check if complex questions can be decomposed to original questions
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "check_origin")
+        check_query_results = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+
+        states = []
+        complex_questions = []
+
+        for idx, result in enumerate(check_query_results):
+            try:
+                if isinstance(result, str):
+                    result = json.loads(self._clean_json_block(result))
+                    if isinstance(result, list):
+                        result = result[0]
+
+                if isinstance(result, dict):
+                    states.append(result.get("state", None))
+                    complex_questions.append(result.get("complex_question", None))
+                else:
+                    self.logger.warning(f"[Skipped]: Invalid result at index {idx}: {result}")
+                    states.append(None)
+                    complex_questions.append(None)
+            except Exception as e:
+                self.logger.warning(f"[Error]: Failed to parse result at index {idx}: {e}")
+                states.append(None)
+                complex_questions.append(None)
+
+        dataframe["state"] = states
+        dataframe[self.output_question_key] = complex_questions
+        dataframe = dataframe[dataframe["state"] == 1].copy()
+
+        # Step 2: Verify if LLM can answer the complex questions
+        sys_prompts, user_prompts = self._reformat_prompt(dataframe, "question_verify")
+        question_verify_results = self.llm_serving.generate_from_input(user_prompts, sys_prompts)
+
+        llm_answers = []
+        for idx, result in enumerate(question_verify_results):
+            try:
+                if isinstance(result, str):
+                    result = json.loads(self._clean_json_block(result))[0]
+
+                if isinstance(result, dict):
+                    llm_answers.append(result.get("llm_answer", None))
+
+                else:
+                    self.logger.warning(f"[Skipped]: Invalid result at index {idx}: {result}")
+                    llm_answers.append(None)
+            except Exception as e:
+                self.logger.warning(f"[Error]: Failed to parse result at index {idx}: {e}")
+                llm_answers.append(None)
+
+        dataframe["llm_answer"] = llm_answers
+        
+        llm_score = self.recall_score(dataframe)
+        dataframe["llm_score"] = llm_score
+        dataframe = dataframe[dataframe["llm_score"] < 1].drop(columns=["llm_score"]).reset_index(drop=True)
+        dataframe = dataframe.drop(columns="llm_answer")
+
+        output_file = storage.write(dataframe)
+        self.logger.info(f"Results saved to {output_file}")
+        return
\ No newline at end of file

dataflow/operators/chemistry/__init__.py

+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from generate.extract_smiles_from_text_generator import ExtractSmilesFromTextGenerator
+    from eval.smiles_equivalence_dataset_evaluator import SmilesEquivalenceDatasetEvaluator
+else:
+    import sys
+    from dataflow.utils.registry import LazyLoader, generate_import_structure_from_type_checking
+
+    cur_path = "dataflow/operators/chemistry/"
+
+    _import_structure = generate_import_structure_from_type_checking(__file__, cur_path)
+    sys.modules[__name__] = LazyLoader(__name__, "dataflow/operators/chemistry/", _import_structure)

dataflow/operators/chemistry/eval/smiles_equivalence_dataset_evaluator.py

+import pandas as pd
+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow import get_logger
+
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+from dataflow.core import LLMServingABC
+import json
+
+@OPERATOR_REGISTRY.register()
+class SmilesEquivalenceDatasetEvaluator(OperatorABC):
+    """
+    对每个块（row）里的 golden_label 与 synth_smiles 进行 SMILES 等价性评估：
+    - 以 abbreviation 对齐
+    - RDKit 规范 SMILES 后比较是否相等（相等=1，否则=0；不存在对应项=0）
+    - 输出到 dataframe 的新列：final_result（list[dict]）、block_score、block_total、block_accuracy
+    - 统计 overall 汇总并保存到 self.overall_summary
+    """
+    def __init__(self, llm_serving: LLMServingABC = None):
+        self.logger = get_logger()
+        self.overall_summary = None  # 评估完成后填充
+
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        if lang == "zh":
+            return (
+                "评估 golden_label 与 synth_smiles 的 SMILES 等价性并计算分数。"
+                "逐块输出 final_result、块内得分与准确率，并统计全局总分。"
+            )
+        elif lang == "en":
+            return (
+                "Evaluate SMILES equivalence between golden_label and synth_smiles per block, "
+                "emit final_result, block-level scores/accuracy, and compute overall summary."
+            )
+        else:
+            return "Evaluate SMILES equivalence for each block."
+
+    # ========= 内部工具函数（不对用户暴露） =========
+    def _are_smiles_equivalent(self, s1: str, s2: str) -> bool:
+        """
+        使用 RDKit 将 SMILES 规范化后判断是否等价。
+        任意一侧解析失败，返回 False。
+        """
+        try:
+            from rdkit import Chem
+        except ImportError:
+            raise Exception(
+                """
+        rdkit is not installed in this environment yet.
+        Please use pip install rdkit.
+        """
+            )
+        try:
+            m1, m2 = Chem.MolFromSmiles(s1), Chem.MolFromSmiles(s2)
+            if m1 is None or m2 is None:
+                return False
+            c1 = Chem.MolToSmiles(m1, canonical=True)
+            c2 = Chem.MolToSmiles(m2, canonical=True)
+            return c1 == c2
+        except Exception as e:
+            self.logger.warning(f"SMILES 解析异常: {e}")
+            return False
+
+    def _score_one_block(self, golden_label, synth_smiles):
+        """
+        对单个块计算：
+        - final_result: list[ {abbreviation, full_name, smiles, score} ]
+        - block_score, block_total, block_accuracy
+        """
+        golden = golden_label or []
+        synth = synth_smiles or []
+
+        # 用 abbreviation 建索引（synth_smiles 可能为空/缺字段）
+        synth_map = {}
+        for item in synth:
+            abbr = (item or {}).get("abbreviation")
+            smi  = (item or {}).get("smiles")
+            if abbr is not None and smi is not None:
+                synth_map[abbr] = smi
+
+        final_result = []
+        block_score = 0
+
+        for g in golden:
+            abbr = (g or {}).get("abbreviation", "")
+            gold_smi = (g or {}).get("smiles", "")
+            synth_smi = synth_map.get(abbr)  # 可能 None
+
+            score = 0
+            if synth_smi is not None:
+                score = 1 if self._are_smiles_equivalent(gold_smi, synth_smi) else 0
+
+            block_score += score
+            final_result.append({
+                "abbreviation": abbr,
+                "full_name": (g or {}).get("full_name", ""),
+                "smiles": gold_smi,
+                "score": score
+            })
+
+        block_total = len(golden)
+        block_accuracy = (block_score / block_total) if block_total else None
+
+        return final_result, block_score, block_total, block_accuracy
+
+    # ========= 对外主流程 =========
+    def run(
+        self,
+        storage: DataFlowStorage,
+        input_golden_key: str = "golden_label",
+        input_synth_key: str = "synth_smiles",
+        output_key: str = "final_result",
+    ):
+        """
+        dataframe 视作读取到的 JSON（每行一个块），按照给定 key 取出 golden/synth，写回评估结果。
+        返回 output_key（供后续算子引用）。
+        """
+        self.logger.info("Running EvaluateSmilesEquivalence...")
+
+        # 读取 dataframe
+        dataframe = storage.read('dataframe')
+        self.logger.info(f"Loaded dataframe with {len(dataframe)} rows")
+
+        # 为结果准备列
+        block_scores = []
+        block_totals = []
+        block_accuracies = []
+        final_results_all = []
+
+        # 逐行（块）评估
+        for idx, row in dataframe.iterrows():
+            golden_label = row.get(input_golden_key, [])  # list[dict]
+            synth_smiles = row.get(input_synth_key, [])   # list[dict]（可能为空）
+
+            final_result, block_score, block_total, block_accuracy = \
+                self._score_one_block(golden_label, synth_smiles)
+
+            final_results_all.append(final_result)
+            block_scores.append(block_score)
+            block_totals.append(block_total)
+            block_accuracies.append(block_accuracy)
+
+        # 写回列
+        dataframe[output_key] = final_results_all
+        dataframe["block_score"] = block_scores
+        dataframe["block_total"] = block_totals
+        dataframe["block_accuracy"] = block_accuracies
+
+        # 汇总总体分数
+        overall_score = int(sum(block_scores))
+        overall_total = int(sum(block_totals))
+        overall_accuracy = (overall_score / overall_total) if overall_total else None
+
+        self.overall_summary = {
+            "overall_score": overall_score,
+            "overall_total": overall_total,
+            "overall_accuracy": overall_accuracy
+        }
+        self.logger.info(f"Overall summary: {self.overall_summary}")
+
+        # 保存更新后的 dataframe
+        storage.write(dataframe)
+
+        # 返回给后续算子使用的列名（与 ExtractSmilesFromText 一致风格）
+        return output_key

dataflow/operators/chemistry/generate/extract_smiles_from_text_generator.py

+import pandas as pd
+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow import get_logger
+
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+from dataflow.core import LLMServingABC
+import json
+
+import json
+import re
+
+from typing import Union
+from dataflow.core.prompt import prompt_restrict, DIYPromptABC
+from dataflow.prompts.chemistry import ExtractSmilesFromTextPrompt
+
+
+@prompt_restrict(
+    ExtractSmilesFromTextPrompt
+)
+@OPERATOR_REGISTRY.register()
+class ExtractSmilesFromTextGenerator(OperatorABC):
+    '''
+    Answer Generator is a class that generates answers for given questions.
+    '''
+    def __init__(self, llm_serving: LLMServingABC, prompt_template: Union[ExtractSmilesFromTextPrompt, DIYPromptABC] = ExtractSmilesFromTextPrompt):
+        self.logger = get_logger()
+        self.llm_serving = llm_serving
+        self.prompt_template = prompt_template
+        self.json_failures = 0
+    
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        if lang == "zh":
+            return (
+                "ExtractSmilesFromText 用于从 OCR 文本中抽取或解析化学分子的 SMILES 表达式。"
+                "算子会根据给定的提示模板（prompt_template），结合文本内容和（可选的）单体缩写信息，"
+                "调用大语言模型完成解析与结构化，并将结果以 JSON 格式写回到指定列。\n"
+                "\n输入参数：\n"
+                "- llm_serving：LLM 服务对象，需实现 LLMServingABC 接口\n"
+                "- prompt_template：提示词模板对象，用于构造模型输入\n"
+                "- input_content_key： OCR 文本的列名（默认 'text'）\n"
+                "- input_abbreviation_key：包含缩写/单体信息的列名（默认 'abbreviations'），可为空\n"
+                "- output_key：写回抽取结果的列名（默认 'synth_smiles'）\n"
+                "\n输出参数：\n"
+                "- DataFrame，其中 output_key 列为模型返回并经 JSON 解析后的 SMILES 结构\n"
+                "- 返回 output_key，供后续算子引用\n"
+                "\n备注：\n"
+                "- 模型输出会尝试解析为 JSON；若解析失败，将返回 [] 并记录失败次数。"
+            )
+        elif lang == "en":
+            return (
+                "ExtractSmilesFromText is designed to extract or parse chemical SMILES expressions "
+                "from OCR text. It uses a given prompt_template to construct model inputs, combining "
+                "text content and optional abbreviation/monomer information, then calls the LLM to "
+                "produce structured outputs which are parsed into JSON.\n"
+                "\nInput Parameters:\n"
+                "- llm_serving: LLM serving object implementing LLMServingABC\n"
+                "- prompt_template: Prompt template object used to build model input\n"
+                "- input_content_key: Column name containing OCR text (default 'text')\n"
+                "- input_abbreviation_key: Column name containing abbreviations/monomer info (default 'abbreviations'); optional\n"
+                "- output_key: Column name to store extracted results (default 'synth_smiles')\n"
+                "\nOutput:\n"
+                "- DataFrame with output_key column containing JSON-parsed SMILES structures\n"
+                "- Returns output_key for downstream operators\n"
+                "\nNotes:\n"
+                "- The operator attempts to parse model outputs as JSON; failures return [] and are counted."
+            )
+        else:
+            return "ExtractSmilesFromText extracts chemical SMILES expressions from OCR text using an LLM."
+
+    def _strip_code_fence(self, s: str) -> str:
+        s = s.strip()
+        # 去掉 ```json ... ``` 或 ``` ... ```
+        if s.startswith("```"):
+            # 去掉第一行的 ```(json)?
+            s = re.sub(r"^```(?:json|JSON)?\s*", "", s)
+            # 去掉结尾 ```
+            s = re.sub(r"\s*```$", "", s)
+        return s.strip()
+
+    def _safe_json_load(self, item):
+        """
+        尝试把 item 解析为 JSON：
+        - 解析失败：返回 []，并将 self.json_failures += 1
+        - 解析成功：返回解析后的对象（list/dict/其他 JSON 标准类型）
+        """
+        try:
+            # 已经是结构化就直接返回
+            if isinstance(item, (list, dict)):
+                return item
+            if item is None:
+                return []  # 约定空返回
+
+            # 其他类型（比如 float/int）转成字符串再处理
+            if not isinstance(item, str):
+                item = str(item)
+
+            s = item.strip()
+            if not s:
+                return []  # 空字符串
+
+            # 去掉代码块围栏
+            s = self._strip_code_fence(s)
+
+            # 去掉包裹的引号（例如整个内容被 "..." 或 '...' 包着）
+            if (s.startswith('"') and s.endswith('"')) or (s.startswith("'") and s.endswith("'")):
+                s = s[1:-1].strip()
+
+            # 去掉前缀 json/JSON
+            s = re.sub(r"^\s*(?:json|JSON)\s*", "", s)
+
+            # 定位第一个 { 或 [，从这里开始截
+            m = re.search(r"[\[\{]", s)
+            if m:
+                s = s[m.start():].strip()
+
+            # 如果末尾有多余字符，尝试保留到最后一个 ] 或 }
+            last_bracket = max(s.rfind("]"), s.rfind("}"))
+            if last_bracket != -1:
+                s = s[:last_bracket + 1].strip()
+
+            # 第一次解析
+            obj = json.loads(s)
+
+            # 如果第一次解析结果仍是字符串，尝试再解一次（处理二次编码）
+            if isinstance(obj, str):
+                try:
+                    obj2 = json.loads(obj)
+                    return obj2
+                except json.JSONDecodeError:
+                    # 二次解析失败不视为致命，返回第一次结果
+                    return obj
+
+            return obj
+
+        except Exception as e:
+            # 任何异常：计数 + 返回空列表
+            self.json_failures += 1
+            # 打印精简预览，避免日志过长
+            preview = ""
+            try:
+                preview = (s if len(s) <= 200 else s[:200] + "...").replace("\n", "\\n")
+            except Exception:
+                preview = "<unavailable>"
+            self.logger.warning(f"[safe_json_load] 解析失败，第{self.json_failures}次；错误：{type(e).__name__}: {e}；预览: {preview}")
+            return []
+
+    def run(self, storage: DataFlowStorage, input_content_key: str = "text", input_abbreviation_key: str = "abbreviations", output_key: str = "synth_smiles"):
+        # self.input_key, self.output_key = input_key, output_key
+        self.logger.info("Running PromptGenerator...")
+
+        # Load the raw dataframe from the input file
+        dataframe = storage.read('dataframe')
+        self.logger.info(f"Loading, number of rows: {len(dataframe)}")
+
+        # Create a list to hold all generated questions and answers
+        llm_inputs = []
+
+        # Prepare LLM inputs by formatting the prompt with raw content from the dataframe
+        for index, row in dataframe.iterrows():
+            content = row.get(input_content_key, '')
+            monomer = row.get(input_abbreviation_key, '')
+            llm_input = self.prompt_template.build_prompt(monomer) + content 
+            llm_inputs.append(llm_input)
+        
+        # Generate the text using the model
+        try:
+            self.logger.info("Generating text using the model...")
+            generated_outputs = self.llm_serving.generate_from_input(llm_inputs)
+            self.logger.info("Text generation completed.")
+        except Exception as e:
+            self.logger.error(f"Error during text generation: {e}")
+            return
+
+        parsed_outputs = []
+        for item in generated_outputs:
+            try:
+                parsed_outputs.append(self._safe_json_load(item))
+            except Exception:
+                parsed_outputs.append([])
+        #parsed_outputs = [self._safe_json_load(item)['chemical_structures'] for item in generated_outputs]
+        # print(parsed_outputs)
+        dataframe[output_key] = parsed_outputs
+
+        # Save the updated dataframe to the output file
+        output_file = storage.write(dataframe)
+        return output_key
+

dataflow/operators/code/__init__.py

+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    # filter
+    from .filter.code_autogen_filter import CodeAutoGeneratedFilter
+    from .filter.code_doc_quality_filter import CodeDocumentQualityFilter
+    from .filter.code_encoded_data_filter import CodeEncodedDataFilter
+    from .filter.code_file_type_content_filter import CodeFileTypeContentFilter
+    from .filter.code_length_filter import CodeLengthSampleFilter
+    from .filter.code_quality_score_filter import CodeQualityScoreFilter
+    from .filter.code_score_filter import CodeGenericScoreFilter
+    from .filter.code_text_composition_filter import CodeTextCompositionFilter
+    
+    # generate
+    from .generate.code_code_to_instruction_generator import CodeCodeToInstructionGenerator
+    from .generate.code_instruction_to_code_generator import CodeInstructionToCodeGenerator
+    from .generate.code_instruction_enhancement import CodeEnhancementInstructionGenerator
+    from .generate.code_gen_instruction import CodeInstructionGenerator
+    # eval
+    from .eval.code_auto_generated_sample_evaluator import CodeAutoGeneratedSampleEvaluator
+    from .eval.code_document_quality_sample_evaluator import CodeDocumentQualitySampleEvaluator
+    from .eval.code_encoded_data_sample_evaluator import CodeEncodedDataSampleEvaluator
+    from .eval.code_length_sample_evaluator import CodeLengthSampleEvaluator
+    from .eval.code_quality_sample_evaluator import CodeQualitySampleEvaluator
+    from .eval.code_sandbox_sample_evaluator import CodeSandboxSampleEvaluator
+    from .eval.code_text_composition_sample_evaluator import CodeTextCompositionSampleEvaluator
+    from .eval.python_executor import (
+        ImageRuntime,
+        DateRuntime,
+        CustomDict,
+        ColorObjectRuntime,
+        PythonExecutor
+    )
+
+else:
+    import sys
+    from dataflow.utils.registry import LazyLoader, generate_import_structure_from_type_checking
+
+    cur_path = "dataflow/operators/code/"
+
+    _import_structure = generate_import_structure_from_type_checking(__file__, cur_path)
+    sys.modules[__name__] = LazyLoader(__name__, "dataflow/operators/code/", _import_structure)

dataflow/operators/code/eval/code_auto_generated_sample_evaluator.py

+import pandas as pd
+from typing import List, Dict, Callable, Optional
+from tqdm import tqdm
+
+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow import get_logger
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+
+@OPERATOR_REGISTRY.register()
+class CodeAutoGeneratedSampleEvaluator(OperatorABC):
+    """
+    CodeAutoGeneratedSampleEvaluator evaluates code samples based on auto-generation
+    markers to provide scores for filtering auto-generated code files.
+    """
+
+    # Common markers for auto-generated code
+    AUTOGEN_PATTERNS = [
+        "auto-generated", "autogenerated", "automatically generated",
+        "generated automatically", "this file is generated"
+    ]
+
+    def __init__(self, is_generated_func: Optional[Callable[[], bool]] = None):
+        """
+        Initialize the operator and set up the logger.
+        
+        Args:
+            is_generated_func: Optional external detection function
+        """
+        self.logger = get_logger()
+        self.logger.info(f'Initializing {self.__class__.__name__}...')
+        self.score_name = 'CodeAutoGeneratedScore'
+        self.is_generated_func = is_generated_func
+        self.logger.info(f'{self.__class__.__name__} initialized.')
+
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        """
+        Provide operator functionality description and parameter documentation.
+        """
+        if lang == "zh":
+            return (
+                "基于自动生成标记评估代码样本，检测文件头部的自动生成标记。\n\n"
+                "评估指标：\n"
+                "- CodeAutoGeneratedMarkerCount: 检测到的自动生成标记数量\n"
+                "- CodeAutoGeneratedScore: 综合自动生成得分 (0-1，1表示非自动生成)\n\n"
+                "输入要求：需要包含'lines'列\n\n"
+                "输出参数：\n"
+                "- CodeAutoGeneratedMarkerCount: 检测到的自动生成标记数量\n"
+                "- CodeAutoGeneratedScore: 综合自动生成得分"
+            )
+        else:
+            return (
+                "Evaluate code samples based on auto-generation markers, detecting auto-generation markers in file headers.\n\n"
+                "Evaluation Metrics:\n"
+                "- CodeAutoGeneratedMarkerCount: Number of detected auto-generation markers\n"
+                "- CodeAutoGeneratedScore: Comprehensive auto-generation score (0-1, 1 means not auto-generated)\n\n"
+                "Input Requirement: Requires 'lines' column\n\n"
+                "Output Parameters:\n"
+                "- CodeAutoGeneratedMarkerCount: Number of detected auto-generation markers\n"
+                "- CodeAutoGeneratedScore: Comprehensive auto-generation score"
+            )
+
+    def _score_func(self, sample):
+        """
+        Calculate auto-generation scores for a single code sample.
+        
+        Args:
+            sample: Dictionary containing 'lines' key
+            
+        Returns:
+            Dictionary containing auto-generation scores
+        """
+        lines = sample.get('lines', [])
+        
+        # Check external function first
+        marker_count = 0
+        if self.is_generated_func and self.is_generated_func():
+            marker_count = 1
+        
+        # Check first 5 lines for markers
+        first_5 = "\n".join(lines[:5]).lower()
+        for pat in self.AUTOGEN_PATTERNS:
+            if pat in first_5:
+                marker_count += 1
+        
+        # Calculate comprehensive score (0-1)
+        score = 1.0 if marker_count == 0 else 0.0
+        
+        return {
+            'CodeAutoGeneratedMarkerCount': marker_count,
+            'CodeAutoGeneratedScore': score
+        }
+
+    def eval(self, dataframe, input_key):
+        """
+        Evaluate auto-generation markers for all samples in the dataframe.
+        
+        Args:
+            dataframe: Input DataFrame
+            input_key: Key containing the sample data
+            
+        Returns:
+            List of score dictionaries
+        """
+        scores_list = []
+        self.logger.info(f"Evaluating {self.score_name}...")
+        
+        for _, row in dataframe.iterrows():
+            sample = row[input_key] if isinstance(row[input_key], dict) else {"lines": row[input_key]}
+            scores = self._score_func(sample)
+            scores_list.append(scores)
+        
+        self.logger.info("Evaluation complete!")
+        return scores_list
+
+    def run(self, storage: DataFlowStorage, input_key: str):
+        """
+        Execute auto-generation marker evaluation operation.
+        
+        Args:
+            storage: Data storage object
+            input_key: Key name for input data
+        """
+        self.input_key = input_key
+        dataframe = storage.read("dataframe")
+        self.logger.info("CodeAutoGeneratedScore ready to evaluate.")
+        
+        scores = self.eval(dataframe, input_key)
+        
+        # Flatten the nested dictionary of scores into the dataframe
+        for idx, score_dict in enumerate(scores):
+            for key, value in score_dict.items():
+                dataframe.at[idx, key] = value
+        
+        storage.write(dataframe)

dataflow/operators/code/eval/code_document_quality_sample_evaluator.py

+import pandas as pd
+import re
+import math
+from typing import List, Dict, Any
+from collections import Counter
+from tqdm import tqdm
+
+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow import get_logger
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+
+@OPERATOR_REGISTRY.register()
+class CodeDocumentQualitySampleEvaluator(OperatorABC):
+    """
+    CodeDocumentQualitySampleEvaluator evaluates code samples based on comprehensive
+    document-level quality metrics to provide scores for filtering low-quality content.
+    """
+
+    def __init__(self, thresholds: Dict[str, Any] = None):
+        """
+        Initialize the operator and set up thresholds.
+        
+        Args:
+            thresholds: Optional thresholds dictionary to override defaults
+        """
+        self.logger = get_logger()
+        self.logger.info(f'Initializing {self.__class__.__name__}...')
+        self.score_name = 'CodeDocumentQualityScore'
+        
+        # Default thresholds for each rule
+        default_thresholds = {
+            'min_num_chars': 1,
+            'max_num_chars': 100000,
+            'min_num_words': 1,
+            'max_num_words': 100000,
+            'max_frac_duplicate_lines': 1.0,
+            'max_frac_duplicate_2gram': 1.0,
+            'max_frac_duplicate_3gram': 1.0,
+            'max_frac_duplicate_4gram': 1.0,
+            'max_frac_duplicate_5gram': 1.0,
+            'max_frac_duplicate_6gram': 1.0,
+            'max_frac_duplicate_7gram': 1.0,
+            'max_frac_duplicate_8gram': 1.0,
+            'max_frac_duplicate_9gram': 1.0,
+            'max_frac_duplicate_10gram': 1.0,
+            'max_frac_curly_bracket': 1.0,
+            'max_frac_all_caps_words': 1.0,
+            'min_entropy_unigram': 0.0,
+        }
+        
+        # Merge with provided thresholds
+        self.thresholds = default_thresholds.copy()
+        if thresholds:
+            self.thresholds.update(thresholds)
+            
+        self.logger.info(f'{self.__class__.__name__} initialized.')
+
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        """
+        Provide operator functionality description and parameter documentation.
+        """
+        if lang == "zh":
+            return (
+                "基于综合文档级质量指标评估代码样本，包括内容长度、重复模式、字符组成和文本熵值。\n\n"
+                "评估指标：\n"
+                "- CodeDocumentQualityCharCount: 字符数\n"
+                "- CodeDocumentQualityWordCount: 词数\n"
+                "- CodeDocumentQualityDuplicateLinesRatio: 重复行比例\n"
+                "- CodeDocumentQualityDuplicateNgramRatio: n-gram重复比例\n"
+                "- CodeDocumentQualityCurlyBracketRatio: 花括号比例\n"
+                "- CodeDocumentQualityAllCapsRatio: 全大写单词比例\n"
+                "- CodeDocumentQualityEntropy: 单字符熵值\n"
+                "- CodeDocumentQualityScore: 综合文档质量得分 (0-1，1表示通过所有质量检查)\n\n"
+                "输入要求：需要包含'text'、'filename'、'language'列\n\n"
+                "输出参数：\n"
+                "- 各种质量指标的数值\n"
+                "- CodeDocumentQualityScore: 综合文档质量得分"
+            )
+        else:
+            return (
+                "Evaluate code samples based on comprehensive document-level quality metrics including content length, repetition patterns, character composition, and text entropy.\n\n"
+                "Evaluation Metrics:\n"
+                "- CodeDocumentQualityCharCount: Character count\n"
+                "- CodeDocumentQualityWordCount: Word count\n"
+                "- CodeDocumentQualityDuplicateLinesRatio: Duplicate lines ratio\n"
+                "- CodeDocumentQualityDuplicateNgramRatio: n-gram repetition ratio\n"
+                "- CodeDocumentQualityCurlyBracketRatio: Curly bracket ratio\n"
+                "- CodeDocumentQualityAllCapsRatio: All-caps words ratio\n"
+                "- CodeDocumentQualityEntropy: Unigram entropy\n"
+                "- CodeDocumentQualityScore: Comprehensive document quality score (0-1, 1 means passes all quality checks)\n\n"
+                "Input Requirement: Requires 'text', 'filename', 'language' columns\n\n"
+                "Output Parameters:\n"
+                "- Various quality metric values\n"
+                "- CodeDocumentQualityScore: Comprehensive document quality score"
+            )
+
+    def _frac_duplicate_ngrams_n(self, text: str, n: int) -> float:
+        """Calculate the fraction of duplicate n-grams in the text"""
+        words = re.findall(r'\b\w+\b', text)
+        if len(words) < n:
+            return 0.0
+        ngrams = [' '.join(words[i:i+n]) for i in range(len(words)-n+1)]
+        ngram2count = Counter(ngrams)
+        count = sum([v for v in ngram2count.values() if v != 1])
+        total = sum([v for v in ngram2count.values()])
+        return count / total if total else 0.0
+
+    def _num_chars(self, text: str) -> int:
+        return len(text)
+
+    def _num_words(self, text: str) -> int:
+        return len(re.findall(r'\w+', text))
+
+    def _frac_duplicate_lines(self, lines: List[str]) -> float:
+        if not lines:
+            return 0.0
+        line2count = Counter([l.strip() for l in lines if l.strip()])
+        count = sum([v for v in line2count.values() if v != 1])
+        total = sum([v for v in line2count.values()])
+        return count / total if total else 0.0
+
+    def _frac_curly_bracket(self, text: str) -> float:
+        total = len(text)
+        if total == 0:
+            return 0.0
+        count = text.count('{') + text.count('}')
+        return count / total
+
+    def _frac_all_caps_words(self, text: str) -> float:
+        words = re.findall(r'\b\w+\b', text)
+        if not words:
+            return 0.0
+        all_caps = [w for w in words if w.isupper() and len(w) > 1]
+        return len(all_caps) / len(words)
+
+    def _entropy_unigram(self, text: str) -> float:
+        words = re.findall(r'\b\w+\b', text)
+        if not words:
+            return 0.0
+        freq = Counter(words)
+        total = sum(freq.values())
+        entropy = -sum((c/total) * math.log2(c/total) for c in freq.values())
+        return entropy
+
+    def _score_func(self, sample):
+        """
+        Calculate document quality scores for a single code sample.
+        
+        Args:
+            sample: Dictionary containing 'text', 'filename', 'language' keys
+            
+        Returns:
+            Dictionary containing document quality scores
+        """
+        text = sample.get('text', sample.get('content', ''))
+        filename = sample.get('filename', '')
+        lang = sample.get('language', 'en')
+        lines = text.splitlines()
+        
+        # Use instance thresholds
+        thresh = self.thresholds
+        
+        # Calculate individual metrics
+        num_chars = self._num_chars(text)
+        num_words = self._num_words(text)
+        frac_dup_lines = self._frac_duplicate_lines(lines)
+        frac_curly = self._frac_curly_bracket(text)
+        frac_caps = self._frac_all_caps_words(text)
+        entropy = self._entropy_unigram(text)
+        
+        # Calculate n-gram ratios
+        ngram_ratios = {}
+        for n in range(2, 11):
+            key = f'max_frac_duplicate_{n}gram'
+            if key in thresh:
+                ngram_ratios[f'duplicate_{n}gram_ratio'] = self._frac_duplicate_ngrams_n(text, n)
+        
+        # Calculate comprehensive score (0-1)
+        score = 1.0
+        
+        # Apply all quality rules
+        if num_chars < thresh['min_num_chars'] or num_chars > thresh['max_num_chars']:
+            score = 0.0
+        elif num_words < thresh['min_num_words'] or num_words > thresh['max_num_words']:
+            score = 0.0
+        elif frac_dup_lines > thresh['max_frac_duplicate_lines']:
+            score = 0.0
+        elif frac_curly > thresh['max_frac_curly_bracket']:
+            score = 0.0
+        elif frac_caps > thresh['max_frac_all_caps_words']:
+            score = 0.0
+        elif entropy < thresh['min_entropy_unigram']:
+            score = 0.0
+        else:
+            # Check n-gram ratios
+            for n in range(2, 11):
+                key = f'max_frac_duplicate_{n}gram'
+                if key in thresh:
+                    frac_dup_ngram = self._frac_duplicate_ngrams_n(text, n)
+                    if frac_dup_ngram > thresh[key]:
+                        score = 0.0
+                        break
+        
+        # Prepare result dictionary
+        result = {
+            'CodeDocumentQualityCharCount': num_chars,
+            'CodeDocumentQualityWordCount': num_words,
+            'CodeDocumentQualityDuplicateLinesRatio': frac_dup_lines,
+            'CodeDocumentQualityCurlyBracketRatio': frac_curly,
+            'CodeDocumentQualityAllCapsRatio': frac_caps,
+            'CodeDocumentQualityEntropy': entropy,
+            'CodeDocumentQualityScore': score
+        }
+        
+        # Add n-gram ratios
+        result.update({f'CodeDocumentQuality{k.title()}Ratio': v for k, v in ngram_ratios.items()})
+        
+        return result
+
+    def eval(self, dataframe, input_key):
+        """
+        Evaluate document quality for all samples in the dataframe.
+        
+        Args:
+            dataframe: Input DataFrame
+            input_key: Key containing the sample data
+            
+        Returns:
+            List of score dictionaries
+        """
+        scores_list = []
+        self.logger.info(f"Evaluating {self.score_name}...")
+        
+        for _, row in dataframe.iterrows():
+            sample = row[input_key] if isinstance(row[input_key], dict) else {"text": row[input_key], "filename": "", "language": "unknown"}
+            scores = self._score_func(sample)
+            scores_list.append(scores)
+        
+        self.logger.info("Evaluation complete!")
+        return scores_list
+
+    def run(self, storage: DataFlowStorage, input_key: str):
+        """
+        Execute document quality evaluation operation.
+        
+        Args:
+            storage: Data storage object
+            input_key: Key name for input data
+        """
+        self.input_key = input_key
+        dataframe = storage.read("dataframe")
+        self.logger.info("CodeDocumentQualityScore ready to evaluate.")
+        
+        scores = self.eval(dataframe, input_key)
+        
+        # Flatten the nested dictionary of scores into the dataframe
+        for idx, score_dict in enumerate(scores):
+            for key, value in score_dict.items():
+                dataframe.at[idx, key] = value
+        
+        storage.write(dataframe)

dataflow/operators/code/eval/code_encoded_data_sample_evaluator.py

+import re
+import pandas as pd
+from typing import Dict
+from tqdm import tqdm
+
+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow import get_logger
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+
+@OPERATOR_REGISTRY.register()
+class CodeEncodedDataSampleEvaluator(OperatorABC):
+    """
+    CodeEncodedDataSampleEvaluator evaluates code samples based on encoded data patterns
+    to provide scores for filtering binary content and auto-generated code.
+    """
+
+    def __init__(self):
+        """
+        Initialize the operator and set up the logger.
+        """
+        self.logger = get_logger()
+        self.logger.info(f'Initializing {self.__class__.__name__}...')
+        self.score_name = 'CodeEncodedDataScore'
+        self.logger.info(f'{self.__class__.__name__} initialized.')
+
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        """
+        Provide operator functionality description and parameter documentation.
+        """
+        if lang == "zh":
+            return (
+                "基于编码数据模式评估代码样本，检测Base64、十六进制和Unicode转义序列。\n\n"
+                "评估指标：\n"
+                "- CodeEncodedDataBase64Ratio: Base64编码数据比例\n"
+                "- CodeEncodedDataHexRatio: 十六进制数据比例\n"
+                "- CodeEncodedDataUnicodeRatio: Unicode转义序列比例\n"
+                "- CodeEncodedDataScore: 综合编码数据得分 (0-1，1表示通过编码数据检查)\n\n"
+                "输入要求：需要包含'text'列\n\n"
+                "输出参数：\n"
+                "- CodeEncodedDataBase64Ratio: Base64编码数据比例\n"
+                "- CodeEncodedDataHexRatio: 十六进制数据比例\n"
+                "- CodeEncodedDataUnicodeRatio: Unicode转义序列比例\n"
+                "- CodeEncodedDataScore: 综合编码数据得分"
+            )
+        else:
+            return (
+                "Evaluate code samples based on encoded data patterns, detecting Base64, hexadecimal, and Unicode escape sequences.\n\n"
+                "Evaluation Metrics:\n"
+                "- CodeEncodedDataBase64Ratio: Base64 encoded data ratio\n"
+                "- CodeEncodedDataHexRatio: Hexadecimal data ratio\n"
+                "- CodeEncodedDataUnicodeRatio: Unicode escape sequence ratio\n"
+                "- CodeEncodedDataScore: Comprehensive encoded data score (0-1, 1 means passes encoded data checks)\n\n"
+                "Input Requirement: Requires 'text' column\n\n"
+                "Output Parameters:\n"
+                "- CodeEncodedDataBase64Ratio: Base64 encoded data ratio\n"
+                "- CodeEncodedDataHexRatio: Hexadecimal data ratio\n"
+                "- CodeEncodedDataUnicodeRatio: Unicode escape sequence ratio\n"
+                "- CodeEncodedDataScore: Comprehensive encoded data score"
+            )
+
+    def _score_func(self, sample):
+        """
+        Calculate encoded data pattern scores for a single code sample.
+        
+        Args:
+            sample: Dictionary containing 'text' key or string
+            
+        Returns:
+            Dictionary containing encoded data scores
+        """
+        if isinstance(sample, str):
+            text = sample
+        else:
+            text = sample.get('text', '')
+        
+        # Compile regular expressions for different data patterns
+        patterns = [
+            re.compile(r"[a-zA-Z0-9+/=\n]{64,}"),  # Base64
+            re.compile(r"(?:\b(?:0x|\\x)?[0-9a-fA-F]{2}(?:,|\b\s*)){8,}"),  # Hex
+            re.compile(r"(?:\\u[0-9a-fA-F]{4}){8,}")  # Unicode
+        ]
+        
+        # Calculate ratios for each pattern
+        base64_ratio = 0.0
+        hex_ratio = 0.0
+        unicode_ratio = 0.0
+        
+        for i, pattern in enumerate(patterns):
+            total_match_len = 0
+            for match in pattern.finditer(text):
+                match_len = len(match.group())
+                total_match_len += match_len
+            
+            ratio = total_match_len / max(1, len(text))
+            
+            if i == 0:  # Base64
+                base64_ratio = ratio
+            elif i == 1:  # Hex
+                hex_ratio = ratio
+            else:  # Unicode
+                unicode_ratio = ratio
+        
+        # Calculate comprehensive score (0-1)
+        score = 1.0
+        
+        # Check if any pattern exceeds thresholds
+        if base64_ratio > 0.5 or hex_ratio > 0.5 or unicode_ratio > 0.5:
+            score = 0.0
+        
+        return {
+            'CodeEncodedDataBase64Ratio': base64_ratio,
+            'CodeEncodedDataHexRatio': hex_ratio,
+            'CodeEncodedDataUnicodeRatio': unicode_ratio,
+            'CodeEncodedDataScore': score
+        }
+
+    def eval(self, dataframe, input_key):
+        """
+        Evaluate encoded data patterns for all samples in the dataframe.
+        
+        Args:
+            dataframe: Input DataFrame
+            input_key: Key containing the sample data
+            
+        Returns:
+            List of score dictionaries
+        """
+        scores_list = []
+        self.logger.info(f"Evaluating {self.score_name}...")
+        
+        for _, row in dataframe.iterrows():
+            text = row[input_key] if isinstance(row[input_key], str) else row[input_key].get('text', '')
+            scores = self._score_func(text)
+            scores_list.append(scores)
+        
+        self.logger.info("Evaluation complete!")
+        return scores_list
+
+    def run(self, storage: DataFlowStorage, input_key: str):
+        """
+        Execute encoded data pattern evaluation operation.
+        
+        Args:
+            storage: Data storage object
+            input_key: Key name for input data
+        """
+        self.input_key = input_key
+        dataframe = storage.read("dataframe")
+        self.logger.info("CodeEncodedDataScore ready to evaluate.")
+        
+        scores = self.eval(dataframe, input_key)
+        
+        # Flatten the nested dictionary of scores into the dataframe
+        for idx, score_dict in enumerate(scores):
+            for key, value in score_dict.items():
+                dataframe.at[idx, key] = value
+        
+        storage.write(dataframe)

dataflow/operators/code/eval/code_length_sample_evaluator.py

+import pandas as pd
+from typing import List
+from tqdm import tqdm
+
+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow import get_logger
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+
+@OPERATOR_REGISTRY.register()
+class CodeLengthSampleEvaluator(OperatorABC):
+    """
+    CodeLengthSampleEvaluator evaluates code samples based on length characteristics
+    to provide scores for filtering oversized files and poorly formatted code.
+    It analyzes total lines, average line length, and maximum line length,
+    applying different thresholds for different programming languages.
+    """
+
+    # List of languages that require special handling (these languages allow longer line lengths)
+    EXCLUDED_LANGS = {
+        "HTML", "JSON", "Markdown", "Roff", "Roff Manpage", 
+        "SMT", "TeX", "Text", "XML"
+    }
+
+    def __init__(self):
+        """
+        Initialize the operator and set up the logger.
+        """
+        self.logger = get_logger()
+        self.logger.info(f'Initializing {self.__class__.__name__}...')
+        self.score_name = 'CodeLengthScore'
+        self.logger.info(f'{self.__class__.__name__} initialized.')
+
+    @staticmethod
+    def get_desc(lang: str = "zh"):
+        """
+        Provide operator functionality description and parameter documentation.
+        """
+        if lang == "zh":
+            return (
+                "基于代码长度特征评估代码样本，分析总行数、平均行长和最大行长。\n\n"
+                "评估指标：\n"
+                "- CodeLengthTotalLines: 总行数\n"
+                "- CodeLengthAvgLineLength: 平均行长\n"
+                "- CodeLengthMaxLineLength: 最大行长\n"
+                "- CodeLengthScore: 综合长度得分 (0-1，1表示通过所有长度检查)\n\n"
+                "输入要求：需要包含'lines'和'language'列\n\n"
+                "输出参数：\n"
+                "- CodeLengthTotalLines: 总行数\n"
+                "- CodeLengthAvgLineLength: 平均行长\n"
+                "- CodeLengthMaxLineLength: 最大行长\n"
+                "- CodeLengthScore: 综合长度得分"
+            )
+        else:
+            return (
+                "Evaluate code samples based on length characteristics, analyzing total lines, average line length, and maximum line length.\n\n"
+                "Evaluation Metrics:\n"
+                "- CodeLengthTotalLines: Total number of lines\n"
+                "- CodeLengthAvgLineLength: Average line length\n"
+                "- CodeLengthMaxLineLength: Maximum line length\n"
+                "- CodeLengthScore: Comprehensive length score (0-1, 1 means passes all length checks)\n\n"
+                "Input Requirement: Requires 'lines' and 'language' columns\n\n"
+                "Output Parameters:\n"
+                "- CodeLengthTotalLines: Total number of lines\n"
+                "- CodeLengthAvgLineLength: Average line length\n"
+                "- CodeLengthMaxLineLength: Maximum line length\n"
+                "- CodeLengthScore: Comprehensive length score"
+            )
+
+    def _score_func(self, sample):
+        """
+        Calculate length-based scores for a single code sample.
+        
+        Args:
+            sample: Dictionary containing 'lines' and 'language' keys
+            
+        Returns:
+            Dictionary containing length scores
+        """
+        lines = sample.get('lines', [])
+        language = sample.get('language', '')
+        
+        # Calculate basic metrics
+        n_lines = len(lines)
+        avg_len = sum(len(l) for l in lines) / max(1, n_lines)
+        max_len = max((len(l) for l in lines), default=0)
+        
+        # Calculate comprehensive score (0-1)
+        score = 1.0
+        
+        # Check total number of lines
+        if n_lines > 100_000:
+            score = 0.0
+        
+        # Apply different rules based on language type
+        elif language not in self.EXCLUDED_LANGS:
+            if avg_len > 100 or max_len > 1000:
+                score = 0.0
+        else:
+            if max_len > 100_000:
+                score = 0.0
+        
+        return {
+            'CodeLengthTotalLines': n_lines,
+            'CodeLengthAvgLineLength': avg_len,
+            'CodeLengthMaxLineLength': max_len,
+            'CodeLengthScore': score
+        }
+
+    def eval(self, dataframe, input_key):
+        """
+        Evaluate length characteristics for all samples in the dataframe.
+        
+        Args:
+            dataframe: Input DataFrame
+            input_key: Key containing the sample data
+            
+        Returns:
+            List of score dictionaries
+        """
+        scores_list = []
+        self.logger.info(f"Evaluating {self.score_name}...")
+        
+        for _, row in dataframe.iterrows():
+            sample = row[input_key] if isinstance(row[input_key], dict) else {"lines": row[input_key], "language": "unknown"}
+            scores = self._score_func(sample)
+            scores_list.append(scores)
+        
+        self.logger.info("Evaluation complete!")
+        return scores_list
+
+    def run(self, storage: DataFlowStorage, input_key: str):
+        """
+        Execute length evaluation operation.
+        
+        Args:
+            storage: Data storage object
+            input_key: Key name for input data
+        """
+        self.input_key = input_key
+        dataframe = storage.read("dataframe")
+        self.logger.info("CodeLengthScore ready to evaluate.")
+        
+        scores = self.eval(dataframe, input_key)
+        
+        # Flatten the nested dictionary of scores into the dataframe
+        for idx, score_dict in enumerate(scores):
+            for key, value in score_dict.items():
+                dataframe.at[idx, key] = value
+        
+        storage.write(dataframe)

dataflow/operators/code/eval/code_quality_sample_evaluator.py

+import pandas as pd
+import re
+from typing import List, Tuple
+
+# Assuming these are the correct import paths for your framework
+from dataflow.utils.registry import OPERATOR_REGISTRY
+from dataflow import get_logger
+from dataflow.utils.storage import DataFlowStorage
+from dataflow.core import OperatorABC
+from dataflow.core import LLMServingABC
+from dataflow.core.prompt import prompt_restrict, DIYPromptABC
+from dataflow.prompts.code import CodeQualityEvaluatorPrompt, DiyCodePrompt
+
+from typing import Union
+@prompt_restrict(
+    CodeQualityEvaluatorPrompt,
+    DiyCodePrompt
+)
+@OPERATOR_REGISTRY.register()
+class CodeQualitySampleEvaluator(OperatorABC):
+    """
+    CodeQualitySampleEvaluator is an operator that evaluates the quality of a generated code snippet
+    against its source instruction. It uses an LLM to provide both a numerical score
+    and textual feedback, acting as an automated code reviewer.
+    """
+
+    def __init__(self, llm_serving: LLMServingABC, prompt_template: Union[CodeQualityEvaluatorPrompt, DiyCodePrompt, DIYPromptABC] = None):
+        """
+        Initializes the operator with a language model serving endpoint.
+        """
+        self.logger = get_logger()
+        self.logger.info(f'Initializing {self.__class__.__name__}...')
+        self.llm_serving = llm_serving
+        self.score_name = 'CodeQualityScore'
+        
+        # Initialize prompt template
+        if prompt_template is None:
+            prompt_template = CodeQualityEvaluatorPrompt()
+        elif isinstance(prompt_template, str):
+            prompt_template = DiyCodePrompt(prompt_template)
+        self.prompt_template = prompt_template
+        
+        self.logger.info(f'{self.__class__.__name__} initialized.')
+    
+    @staticmethod
+    def get_desc(lang: str = "en"):
+        """
+        Provides a description of the operator's function and parameters.
+        """
+        if lang == "zh":
+            return (
+                "该算子用于评估生成的代码片段与其源指令的匹配质量，并输出分数和反馈。\n\n"
+                "输入参数：\n"
+                "- input_instruction_key: 包含人类指令的字段名 (默认: 'generated_instruction')\n"
+                "- input_code_key: 包含生成代码的字段名 (默认: 'generated_code')\n"
+                "输出参数：\n"
+                "- output_score_key: 用于存储质量分数的字段名 (默认: 'quality_score')\n"
+                "- output_feedback_key: 用于存储质量反馈的字段名 (默认: 'quality_feedback')\n"
+            )
+        else: # Default to English
+            return (
+                "This operator evaluates the quality of a generated code snippet against its source instruction, providing a score and feedback.\n\n"
+                "Input Parameters:\n"
+                "- input_instruction_key: Field name containing the human instruction (default: 'generated_instruction')\n"
+                "- input_code_key: Field name containing the generated code (default: 'generated_code')\n"
+                "Output Parameters:\n"
+                "- output_score_key: Field name to store the quality score (default: 'quality_score')\n"
+                "- output_feedback_key: Field name to store the quality feedback (default: 'quality_feedback')\n"
+            )
+
+    def _validate_dataframe(self, dataframe: pd.DataFrame):
+        """
+        Validates the DataFrame to ensure required columns exist and output columns don't.
+        """
+        required_keys = [self.input_key]
+        forbidden_keys = [self.output_score_key, self.output_feedback_key]
+
+        missing = [k for k in required_keys if k not in dataframe.columns]
+        conflict = [k for k in forbidden_keys if k in dataframe.columns]
+
+        if missing:
+            raise ValueError(f"Missing required column(s) for CodeQualitySampleEvaluator: {missing}")
+        if conflict:
+            raise ValueError(f"The following column(s) already exist and would be overwritten by CodeQualitySampleEvaluator: {conflict}")
+
+    def _score_func(self, instruction: str, code: str) -> Tuple[int, str]:
+        """
+        Evaluate a single instruction-code pair and return score and feedback.
+        
+        Args:
+            instruction: The instruction text
+            code: The generated code text
+            
+        Returns:
+            Tuple of (score, feedback) where score is an integer and feedback is a string
+        """
+        prompt = self.prompt_template.build_prompt(instruction=instruction, code=code)
+        response = self.llm_serving.generate_from_input(user_inputs=[prompt], system_prompt="")
+        
+        if not response or len(response) == 0:
+            self.logger.warning("Empty response from LLM")
+            return 0, "No response from LLM"
+            
+        return self._parse_score_and_feedback(response[0])
+    
+    def _parse_score_and_feedback(self, response: str) -> Tuple[int, str]:
+        """
+        Parse the LLM's raw response to extract the score and feedback.
+        Handles potential formatting errors gracefully.
+        
+        Args:
+            response: Raw response string from the LLM
+            
+        Returns:
+            Tuple of (score, feedback) where score is an integer and feedback is a string
+        """
+        try:
+            score_match = re.search(r"Score:\s*(\d+)", response)
+            feedback_match = re.search(r"Feedback:\s*(.*)", response, re.DOTALL)
+            
+            score = int(score_match.group(1)) if score_match else 0
+            feedback = feedback_match.group(1).strip() if feedback_match else "No feedback provided."
+            
+            return score, feedback
+        except (AttributeError, ValueError, IndexError):
+            # If parsing fails for any reason, return default error values
+            self.logger.warning(f"Failed to parse LLM evaluation output: '{response}'")
+            return 0, "Failed to parse LLM evaluation output."
+
+    def eval(self, dataframe: pd.DataFrame, input_instruction_key: str, input_code_key: str) -> Tuple[List[int], List[str]]:
+        """
+        Evaluate instruction-code pairs and return scores and feedbacks.
+        
+        Args:
+            dataframe: Input DataFrame
+            input_key: Field name containing instruction-code pairs (as dict with 'instruction' and 'code' keys)
+            
+        Returns:
+            Tuple of (scores, feedbacks) lists
+        """
+        self.logger.info(f"Evaluating {self.score_name}...")
+        
+        scores = []
+        feedbacks = []
+        
+        for _, row in dataframe.iterrows():
+            instruction = row[input_instruction_key]
+            code = row[input_code_key]
+
+            score, feedback = self._score_func(instruction, code)
+            scores.append(score)
+            feedbacks.append(feedback)
+        
+        self.logger.info("Evaluation complete!")
+        return scores, feedbacks
+    
+    def run(
+        self, 
+        storage: DataFlowStorage, 
+        input_instruction_key: str,
+        input_code_key: str,
+        output_score_key: str = "quality_score",
+        output_feedback_key: str = "quality_feedback"
+    ):
+        """
+        Executes the scoring process for instruction-code pairs.
+        
+        Args:
+            storage: Data storage object
+            input_key: Field name containing instruction-code pairs
+            output_score_key: Field name for quality scores
+            output_feedback_key: Field name for quality feedback
+        """
+        self.input_instruction_key = input_instruction_key
+        self.input_code_key = input_code_key
+        self.output_score_key = output_score_key
+        self.output_feedback_key = output_feedback_key
+        
+        dataframe = storage.read("dataframe")
+        scores, feedbacks = self.eval(dataframe, input_instruction_key, input_code_key)
+        
+        dataframe[self.output_score_key] = scores
+        dataframe[self.output_feedback_key] = feedbacks
+        storage.write(dataframe)
\ No newline at end of file