README_zh-CN.md 14.7 KB
Newer Older
gaotongxiao's avatar
gaotongxiao committed
1
<div align="center">
Tong Gao's avatar
Tong Gao committed
2
3
4
  <img src="docs/zh_cn/_static/image/logo.svg" width="500px"/>
  <br />
  <br />
gaotongxiao's avatar
gaotongxiao committed
5

Hubert's avatar
Hubert committed
6
[![docs](https://readthedocs.org/projects/opencompass/badge)](https://opencompass.readthedocs.io/zh_CN)
Songyang Zhang's avatar
Songyang Zhang committed
7
[![license](https://img.shields.io/github/license/InternLM/opencompass.svg)](https://github.com/open-compass/opencompass/blob/main/LICENSE)
Tong Gao's avatar
Tong Gao committed
8

Hubert's avatar
Hubert committed
9
<!-- [![PyPI](https://badge.fury.io/py/opencompass.svg)](https://pypi.org/project/opencompass/) -->
gaotongxiao's avatar
gaotongxiao committed
10

Tong Gao's avatar
Tong Gao committed
11
[🌐Website](https://opencompass.org.cn/) |
Ezra-Yu's avatar
Ezra-Yu committed
12
[📘Documentation](https://opencompass.readthedocs.io/zh_CN/latest/index.html) |
13
[🛠️Installation](https://opencompass.readthedocs.io/zh_CN/latest/get_started/installation.html) |
Songyang Zhang's avatar
Songyang Zhang committed
14
[🤔Reporting Issues](https://github.com/open-compass/opencompass/issues/new/choose)
gaotongxiao's avatar
gaotongxiao committed
15
16
17
18
19

[English](/README.md) | 简体中文

</div>

20
<p align="center">
21
    👋 加入我们的 <a href="https://discord.gg/KKwfEbFj7U" target="_blank">Discord</a><a href="https://r.vansin.top/?r=opencompass" target="_blank">微信社区</a>
22
23
</p>

Songyang Zhang's avatar
Songyang Zhang committed
24
25
26
27
28
29
30
31
32
33
## 📣 2023 年度榜单计划

我们有幸与社区共同见证了通用人工智能在过去一年里的巨大进展,也非常高兴OpenCompass能够帮助广大大模型开发者和使用者。

我们宣布将启动**OpenCompass 2023年度大模型榜单**发布计划。我们预计将于2024年1月发布大模型年度榜单,系统性评估大模型在语言、知识、推理、创作、长文本和智能体等多个能力维度的表现。

届时,我们将发布开源模型和商业API模型能力榜单,以期为业界提供一份**全面、客观、中立**的参考。

我们诚挚邀请各类大模型接入OpenCompass评测体系,以展示其在各个领域的性能优势。同时,也欢迎广大研究者、开发者向我们提供宝贵的意见和建议,共同推动大模型领域的发展。如有任何问题或需求,请随时[联系我们](mailto:opencompass@pjlab.org.cn)。此外,相关评测内容,性能数据,评测方法也将随榜单发布一并开源。

34
35
我们提供了本次评测所使用的部分题目示例,详情请见[CompassBench 2023](docs/zh_cn/advanced_guides/compassbench_intro.md).

Songyang Zhang's avatar
Songyang Zhang committed
36
37
<p>让我们共同期待OpenCompass 2023年度大模型榜单的发布,期待各大模型在榜单上的精彩表现!</p>

Songyang Zhang's avatar
Songyang Zhang committed
38
39
40
## 🧭	欢迎

来到**OpenCompass**
Tong Gao's avatar
Tong Gao committed
41
42
43

就像指南针在我们的旅程中为我们导航一样,我们希望OpenCompass能够帮助你穿越评估大型语言模型的重重迷雾。OpenCompass提供丰富的算法和功能支持,期待OpenCompass能够帮助社区更便捷地对NLP模型的性能进行公平全面的评估。

Songyang Zhang's avatar
Songyang Zhang committed
44
45
🚩🚩🚩 欢迎加入 OpenCompass!我们目前**招聘全职研究人员/工程师和实习生**。如果您对 LLM 和 OpenCompass 充满热情,请随时通过[电子邮件](mailto:zhangsongyang@pjlab.org.cn)与我们联系。我们非常期待与您交流!

46
47
48
🔥🔥🔥 祝贺 **OpenCompass 作为大模型标准测试工具被Meta AI官方推荐**, 点击 Llama 的 [入门文档](https://ai.meta.com/llama/get-started/#validation) 获取更多信息.

> **注意**<br />
Songyang Zhang's avatar
Songyang Zhang committed
49
> 我们正式启动 OpenCompass 共建计划,诚邀社区用户为 OpenCompass 提供更具代表性和可信度的客观评测数据集!
Songyang Zhang's avatar
Songyang Zhang committed
50
> 点击 [Issue](https://github.com/open-compass/opencompass/issues/248) 获取更多数据集.
Songyang Zhang's avatar
Songyang Zhang committed
51
52
> 让我们携手共进,打造功能强大易用的大模型评测平台!

Songyang Zhang's avatar
Songyang Zhang committed
53
## 🚀 最新进展 <a><img width="35" height="20" src="https://user-images.githubusercontent.com/12782558/212848161-5e783dd6-11e8-4fe0-bbba-39ffb77730be.png"></a>
Leymore's avatar
Leymore committed
54

Fengzhe Zhou's avatar
Fengzhe Zhou committed
55
56
- **\[2024.01.17\]** 我们支持了 [InternLM2](https://github.com/open-compass/opencompass/blob/main/configs/eval_internlm2_chat_keyset.py)[InternLM2-Chat](https://github.com/open-compass/opencompass/blob/main/configs/eval_internlm2_chat_keyset.py) 的相关评测,InternLM2 在这些测试中表现出非常强劲的性能,欢迎试用!🔥🔥🔥.
- **\[2024.01.17\]** 我们支持了多根针版本的大海捞针测试,更多信息见[这里](https://opencompass.readthedocs.io/zh-cn/latest/advanced_guides/needleinahaystack_eval.html#id8)🔥🔥🔥.
57
- **\[2023.12.28\]** 我们支持了对使用[LLaMA2-Accessory](https://github.com/Alpha-VLLM/LLaMA2-Accessory)(一款强大的LLM开发工具箱)开发的所有模型的无缝评估! 🔥🔥🔥.
58
- **\[2023.12.22\]** 我们开源了[T-Eval](https://github.com/open-compass/T-Eval)用于评测大语言模型工具调用能力。欢迎访问T-Eval的官方[Leaderboard](https://open-compass.github.io/T-Eval/leaderboard.html)获取更多信息! 🔥🔥🔥.
Fengzhe Zhou's avatar
Fengzhe Zhou committed
59
60
- **\[2023.12.10\]** 我们开源了多模评测框架 [VLMEvalKit](https://github.com/open-compass/VLMEvalKit),目前已支持 20+ 个多模态大模型与包括 MMBench 系列在内的 7 个多模态评测集.
- **\[2023.12.10\]** 我们已经支持了Mistral AI的MoE模型 **Mixtral-8x7B-32K**。欢迎查阅[MixtralKit](https://github.com/open-compass/MixtralKit)以获取更多关于推理和评测的详细信息.
Songyang Zhang's avatar
Songyang Zhang committed
61
62

> [更多](docs/zh_cn/notes/news.md)
Leymore's avatar
Leymore committed
63

Songyang Zhang's avatar
Songyang Zhang committed
64
## ✨ 介绍
gaotongxiao's avatar
gaotongxiao committed
65

66
67
![image](https://github.com/open-compass/opencompass/assets/22607038/30bcb2e2-3969-4ac5-9f29-ad3f4abb4f3b)

Tong Gao's avatar
Tong Gao committed
68
69
70
71
OpenCompass 是面向大模型评测的一站式平台。其主要特点如下:

- **开源可复现**:提供公平、公开、可复现的大模型评测方案

Leymore's avatar
Leymore committed
72
- **全面的能力维度**:五大维度设计,提供 70+ 个数据集约 40 万题的的模型评测方案,全面评估模型能力
Tong Gao's avatar
Tong Gao committed
73
74
75
76
77
78
79
80
81

- **丰富的模型支持**:已支持 20+ HuggingFace 及 API 模型

- **分布式高效评测**:一行命令实现任务分割和分布式评测,数小时即可完成千亿模型全量评测

- **多样化评测范式**:支持零样本、小样本及思维链评测,结合标准型或对话型提示词模板,轻松激发各种模型最大性能

- **灵活化拓展**:想增加新模型或数据集?想要自定义更高级的任务分割策略,甚至接入新的集群管理系统?OpenCompass 的一切均可轻松扩展!

Songyang Zhang's avatar
Songyang Zhang committed
82
## 📊 性能榜单
Tong Gao's avatar
Tong Gao committed
83

84
我们将陆续提供开源模型和API模型的具体性能榜单,请见 [OpenCompass Leaderboard](https://opencompass.org.cn/rank) 。如需加入评测,请提供模型仓库地址或标准的 API 接口至邮箱  `opencompass@pjlab.org.cn`.
Tong Gao's avatar
Tong Gao committed
85

Songyang Zhang's avatar
Songyang Zhang committed
86
<p align="right"><a href="#top">🔝返回顶部</a></p>
Tong Gao's avatar
Tong Gao committed
87

Leymore's avatar
Leymore committed
88
89
90
91
## 🛠️ 安装

下面展示了快速安装以及准备数据集的步骤。

92
93
94
95
96
### 💻 环境配置

#### 面向开源模型的GPU环境

```bash
Leymore's avatar
Leymore committed
97
98
99
100
101
conda create --name opencompass python=3.10 pytorch torchvision pytorch-cuda -c nvidia -c pytorch -y
conda activate opencompass
git clone https://github.com/open-compass/opencompass opencompass
cd opencompass
pip install -e .
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
```

#### 面向API模型测试的CPU环境

```bash
conda create -n opencompass python=3.10 pytorch torchvision torchaudio cpuonly -c pytorch -y
conda activate opencompass
git clone https://github.com/open-compass/opencompass opencompass
cd opencompass
pip install -e .
# 如果需要使用各个API模型,请 `pip install -r requirements/api.txt` 安装API模型的相关依赖
```

### 📂 数据准备

```bash
Leymore's avatar
Leymore committed
118
# 下载数据集到 data/ 处
119
120
wget https://github.com/open-compass/opencompass/releases/download/0.2.2.rc1/OpenCompassData-core-20240207.zip
unzip OpenCompassData-core-20240207.zip
Leymore's avatar
Leymore committed
121
122
```

123
有部分第三方功能,如 Humaneval 以及 Llama,可能需要额外步骤才能正常运行,详细步骤请参考[安装指南](https://opencompass.readthedocs.io/zh_CN/latest/get_started/installation.html)
Leymore's avatar
Leymore committed
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154

<p align="right"><a href="#top">🔝返回顶部</a></p>

## 🏗️ ️评测

确保按照上述步骤正确安装 OpenCompass 并准备好数据集后,可以通过以下命令评测 LLaMA-7b 模型在 MMLU 和 C-Eval 数据集上的性能:

```bash
python run.py --models hf_llama_7b --datasets mmlu_ppl ceval_ppl
```

OpenCompass 预定义了许多模型和数据集的配置,你可以通过 [工具](./docs/zh_cn/tools.md#ListConfigs) 列出所有可用的模型和数据集配置。

```bash
# 列出所有配置
python tools/list_configs.py
# 列出所有跟 llama 及 mmlu 相关的配置
python tools/list_configs.py llama mmlu
```

你也可以通过命令行去评测其它 HuggingFace 模型。同样以 LLaMA-7b 为例:

```bash
python run.py --datasets ceval_ppl mmlu_ppl \
--hf-path huggyllama/llama-7b \  # HuggingFace 模型地址
--model-kwargs device_map='auto' \  # 构造 model 的参数
--tokenizer-kwargs padding_side='left' truncation='left' use_fast=False \  # 构造 tokenizer 的参数
--max-out-len 100 \  # 最长生成 token 数
--max-seq-len 2048 \  # 模型能接受的最大序列长度
--batch-size 8 \  # 批次大小
--no-batch-padding \  # 不打开 batch padding,通过 for loop 推理,避免精度损失
Tong Gao's avatar
Tong Gao committed
155
--num-gpus 1  # 运行该模型所需的最少 gpu 数
Leymore's avatar
Leymore committed
156
157
```

Tong Gao's avatar
Tong Gao committed
158
159
160
> **注意**<br />
> 若需要运行上述命令,你需要删除所有从 `# ` 开始的注释。

161
通过命令行或配置文件,OpenCompass 还支持评测 API 或自定义模型,以及更多样化的评测策略。请阅读[快速开始](https://opencompass.readthedocs.io/zh_CN/latest/get_started/quick_start.html)了解如何运行一个评测任务。
Leymore's avatar
Leymore committed
162
163
164
165
166

更多教程请查看我们的[文档](https://opencompass.readthedocs.io/zh_CN/latest/index.html)

<p align="right"><a href="#top">🔝返回顶部</a></p>

Songyang Zhang's avatar
Songyang Zhang committed
167
## 📖 数据集支持
Tong Gao's avatar
Tong Gao committed
168
169
170
171
172
173
174
175
176
177
178
179
180
181

<table align="center">
  <tbody>
    <tr align="center" valign="bottom">
      <td>
        <b>语言</b>
      </td>
      <td>
        <b>知识</b>
      </td>
      <td>
        <b>推理</b>
      </td>
      <td>
Leymore's avatar
Leymore committed
182
        <b>考试</b>
Tong Gao's avatar
Tong Gao committed
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
      </td>
    </tr>
    <tr valign="top">
      <td>
<details open>
<summary><b>字词释义</b></summary>

- WiC
- SummEdits

</details>

<details open>
<summary><b>成语习语</b></summary>

- CHID

</details>

<details open>
<summary><b>语义相似度</b></summary>

- AFQMC
- BUSTM

</details>

<details open>
<summary><b>指代消解</b></summary>

- CLUEWSC
- WSC
- WinoGrande

</details>

<details open>
<summary><b>翻译</b></summary>

- Flores
Leymore's avatar
Leymore committed
223
- IWSLT2017
Tong Gao's avatar
Tong Gao committed
224
225

</details>
Leymore's avatar
Leymore committed
226

Tong Gao's avatar
Tong Gao committed
227
<details open>
Leymore's avatar
Leymore committed
228
<summary><b>多语种问答</b></summary>
Tong Gao's avatar
Tong Gao committed
229

Leymore's avatar
Leymore committed
230
231
- TyDi-QA
- XCOPA
Tong Gao's avatar
Tong Gao committed
232
233
234
235

</details>

<details open>
Leymore's avatar
Leymore committed
236
<summary><b>多语种总结</b></summary>
Tong Gao's avatar
Tong Gao committed
237

Leymore's avatar
Leymore committed
238
239
240
241
242
243
244
245
246
247
248
249
- XLSum

</details>
      </td>
      <td>
<details open>
<summary><b>知识问答</b></summary>

- BoolQ
- CommonSenseQA
- NaturalQuestions
- TriviaQA
Tong Gao's avatar
Tong Gao committed
250
251
252
253
254
255
256
257
258
259
260
261
262
263

</details>
      </td>
      <td>
<details open>
<summary><b>文本蕴含</b></summary>

- CMNLI
- OCNLI
- OCNLI_FC
- AX-b
- AX-g
- CB
- RTE
Leymore's avatar
Leymore committed
264
- ANLI
Tong Gao's avatar
Tong Gao committed
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291

</details>

<details open>
<summary><b>常识推理</b></summary>

- StoryCloze
- COPA
- ReCoRD
- HellaSwag
- PIQA
- SIQA

</details>

<details open>
<summary><b>数学推理</b></summary>

- MATH
- GSM8K

</details>

<details open>
<summary><b>定理应用</b></summary>

- TheoremQA
Leymore's avatar
Leymore committed
292
293
- StrategyQA
- SciBench
Tong Gao's avatar
Tong Gao committed
294
295
296
297
298
299
300
301
302
303
304
305
306
307

</details>

<details open>
<summary><b>综合推理</b></summary>

- BBH

</details>
      </td>
      <td>
<details open>
<summary><b>初中/高中/大学/职业考试</b></summary>

Leymore's avatar
Leymore committed
308
- C-Eval
Tong Gao's avatar
Tong Gao committed
309
310
311
- AGIEval
- MMLU
- GAOKAO-Bench
312
- CMMLU
Tong Gao's avatar
Tong Gao committed
313
- ARC
Leymore's avatar
Leymore committed
314
315
316
317
318
319
320
321
- Xiezhi

</details>

<details open>
<summary><b>医学考试</b></summary>

- CMB
Tong Gao's avatar
Tong Gao committed
322
323
324

</details>
      </td>
Leymore's avatar
Leymore committed
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
    </tr>
</td>
    </tr>
  </tbody>
  <tbody>
    <tr align="center" valign="bottom">
      <td>
        <b>理解</b>
      </td>
      <td>
        <b>长文本</b>
      </td>
      <td>
        <b>安全</b>
      </td>
      <td>
        <b>代码</b>
      </td>
    </tr>
    <tr valign="top">
Tong Gao's avatar
Tong Gao committed
345
346
347
348
349
350
351
352
353
      <td>
<details open>
<summary><b>阅读理解</b></summary>

- C3
- CMRC
- DRCD
- MultiRC
- RACE
Leymore's avatar
Leymore committed
354
355
356
- DROP
- OpenBookQA
- SQuAD2.0
Tong Gao's avatar
Tong Gao committed
357
358
359
360
361
362
363
364
365

</details>

<details open>
<summary><b>内容总结</b></summary>

- CSL
- LCSTS
- XSum
Leymore's avatar
Leymore committed
366
- SummScreen
Tong Gao's avatar
Tong Gao committed
367
368
369
370
371
372
373
374
375
376

</details>

<details open>
<summary><b>内容分析</b></summary>

- EPRSTMT
- LAMBADA
- TNEWS

Leymore's avatar
Leymore committed
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
</details>
      </td>
      <td>
<details open>
<summary><b>长文本理解</b></summary>

- LEval
- LongBench
- GovReports
- NarrativeQA
- Qasper

</details>
      </td>
      <td>
<details open>
<summary><b>安全</b></summary>

- CivilComments
- CrowsPairs
- CValues
- JigsawMultilingual
- TruthfulQA

</details>
<details open>
<summary><b>健壮性</b></summary>

- AdvGLUE

</details>
      </td>
      <td>
<details open>
<summary><b>代码</b></summary>

- HumanEval
- HumanEvalX
- MBPP
- APPs
- DS1000

Tong Gao's avatar
Tong Gao committed
419
420
421
422
423
424
425
426
</details>
      </td>
    </tr>
</td>
    </tr>
  </tbody>
</table>

Songyang Zhang's avatar
Songyang Zhang committed
427
428
429
<p align="right"><a href="#top">🔝返回顶部</a></p>

## 📖 模型支持
gaotongxiao's avatar
gaotongxiao committed
430

Tong Gao's avatar
Tong Gao committed
431
432
433
434
<table align="center">
  <tbody>
    <tr align="center" valign="bottom">
      <td>
Songyang Zhang's avatar
Songyang Zhang committed
435
        <b>开源模型</b>
Tong Gao's avatar
Tong Gao committed
436
437
438
439
      </td>
      <td>
        <b>API 模型</b>
      </td>
Songyang Zhang's avatar
Songyang Zhang committed
440
      <!-- <td>
Tong Gao's avatar
Tong Gao committed
441
        <b>自定义模型</b>
Songyang Zhang's avatar
Songyang Zhang committed
442
      </td> -->
Tong Gao's avatar
Tong Gao committed
443
444
445
    </tr>
    <tr valign="top">
      <td>
gaotongxiao's avatar
gaotongxiao committed
446

447
448
449
450
451
452
453
454
455
456
457
- [InternLM](https://github.com/InternLM/InternLM)
- [LLaMA](https://github.com/facebookresearch/llama)
- [Vicuna](https://github.com/lm-sys/FastChat)
- [Alpaca](https://github.com/tatsu-lab/stanford_alpaca)
- [Baichuan](https://github.com/baichuan-inc)
- [WizardLM](https://github.com/nlpxucan/WizardLM)
- [ChatGLM2](https://github.com/THUDM/ChatGLM2-6B)
- [ChatGLM3](https://github.com/THUDM/ChatGLM3-6B)
- [TigerBot](https://github.com/TigerResearch/TigerBot)
- [Qwen](https://github.com/QwenLM/Qwen)
- [BlueLM](https://github.com/vivo-ai-lab/BlueLM)
Tong Gao's avatar
Tong Gao committed
458
- ……
gaotongxiao's avatar
gaotongxiao committed
459

Tong Gao's avatar
Tong Gao committed
460
461
</td>
<td>
gaotongxiao's avatar
gaotongxiao committed
462

Songyang Zhang's avatar
Songyang Zhang committed
463
- OpenAI
Leymore's avatar
Leymore committed
464
- Claude
465
466
467
468
469
470
471
472
473
- ZhipuAI(ChatGLM)
- Baichuan
- ByteDance(YunQue)
- Huawei(PanGu)
- 360
- Baidu(ERNIEBot)
- MiniMax(ABAB-Chat)
- SenseTime(nova)
- Xunfei(Spark)
Tong Gao's avatar
Tong Gao committed
474
- ……
gaotongxiao's avatar
gaotongxiao committed
475

Tong Gao's avatar
Tong Gao committed
476
</td>
gaotongxiao's avatar
gaotongxiao committed
477

Tong Gao's avatar
Tong Gao committed
478
479
480
</tr>
  </tbody>
</table>
gaotongxiao's avatar
gaotongxiao committed
481

Songyang Zhang's avatar
Songyang Zhang committed
482
483
<p align="right"><a href="#top">🔝返回顶部</a></p>

Songyang Zhang's avatar
Songyang Zhang committed
484
485
486
487
488
## 🔜 路线图

- [ ] 主观评测
  - [ ] 发布主观评测榜单
  - [ ] 发布主观评测数据集
489
- [x] 长文本
Songyang Zhang's avatar
Songyang Zhang committed
490
491
492
493
  - [ ] 支持广泛的长文本评测集
  - [ ] 发布长文本评测榜单
- [ ] 代码能力
  - [ ] 发布代码能力评测榜单
494
  - [x] 提供非Python语言的评测服务
Songyang Zhang's avatar
Songyang Zhang committed
495
496
497
- [ ] 智能体
  - [ ] 支持丰富的智能体方案
  - [ ] 提供智能体评测榜单
498
499
- [x] 鲁棒性
  - [x] 支持各类攻击方法
Songyang Zhang's avatar
Songyang Zhang committed
500

501
502
503
504
## 👷‍♂️ 贡献

我们感谢所有的贡献者为改进和提升 OpenCompass 所作出的努力。请参考[贡献指南](https://opencompass.readthedocs.io/zh_CN/latest/notes/contribution_guide.html)来了解参与项目贡献的相关指引。

Songyang Zhang's avatar
Songyang Zhang committed
505
## 🤝 致谢
gaotongxiao's avatar
gaotongxiao committed
506
507
508

该项目部分的代码引用并修改自 [OpenICL](https://github.com/Shark-NLP/OpenICL)

Leymore's avatar
Leymore committed
509
510
该项目部分的数据集和提示词实现修改自 [chain-of-thought-hub](https://github.com/FranxYao/chain-of-thought-hub), [instruct-eval](https://github.com/declare-lab/instruct-eval)

Songyang Zhang's avatar
Songyang Zhang committed
511
## 🖊️ 引用
gaotongxiao's avatar
gaotongxiao committed
512
513
514
515
516

```bibtex
@misc{2023opencompass,
    title={OpenCompass: A Universal Evaluation Platform for Foundation Models},
    author={OpenCompass Contributors},
Songyang Zhang's avatar
Songyang Zhang committed
517
    howpublished = {\url{https://github.com/open-compass/opencompass}},
gaotongxiao's avatar
gaotongxiao committed
518
519
520
    year={2023}
}
```
Songyang Zhang's avatar
Songyang Zhang committed
521
522

<p align="right"><a href="#top">🔝返回顶部</a></p>