starlight-apk/Qwen
1
0
Fork 0
mirror of https://github.com/QwenLM/Qwen.git synced 2026-05-20 16:35:47 +08:00
Code Issues Packages Projects Releases Wiki Activity

add CMMLU evaluation results

Browse Source
This commit is contained in:
Haonan Li
2023-08-13 20:58:52 +04:00
parent 460ea3418b
commit e7072a49c0
4 changed files with 310 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
README_CN.md
View File

@@ -33,20 +33,20 @@
## 评测表现
Qwen-7B在多个全面评估自然语言理解与生成、数学运算解题、代码生成等能力的评测数据集上，包括MMLU、C-Eval、GSM8K、HumanEval、WMT22等，均超出了同规模大语言模型的表现，甚至超出了如12-13B参数等更大规模的语言模型。
Qwen-7B在多个全面评估自然语言理解与生成、数学运算解题、代码生成等能力的评测数据集上，包括MMLU、C-Eval、GSM8K、HumanEval、WMT22、CMMLU等，均超出了同规模大语言模型的表现，甚至超出了如12-13B参数等更大规模的语言模型。
| Model | MMLU | C-Eval | GSM8K | HumanEval | WMT22 (en-zh) |
| :---------------- | :------------: | :------------: | :------------: | :------------: | :------------: |
| LLaMA-7B | 35.1 | - | 11.0 | 10.5 | 8.7 |
| LLaMA 2-7B | 45.3 | - | 14.6 | 12.8 | 17.9 |
| Baichuan-7B | 42.3 | 42.8 | 9.7 | 9.2 | 26.6 |
| ChatGLM2-6B | 47.9 | 51.7 | 32.4 | 9.2 | - |
| InternLM-7B | 51.0 | 52.8 | 31.2 | 10.4 | 14.8 |
| Baichuan-13B | 51.6 | 53.6 | 26.6 | 12.8 | 30.0 |
| LLaMA-13B | 46.9 | 35.5 | 17.8 | 15.8 | 12.0 |
| LLaMA 2-13B | 54.8 | - | 28.7 | 18.3 | 24.2 |
| ChatGLM2-12B | 56.2 | **61.6** | 40.9 | - | - |
| **Qwen-7B** | **56.7** | 59.6 | **51.6** | **24.4** | **30.6** |
| Model | MMLU | C-Eval | GSM8K | HumanEval | WMT22 (en-zh) | CMMLU |
| :---------------- | :------------: | :------------: | :------------: | :------------: | :------------: |:------------: |
| LLaMA-7B | 35.1 | - | 11.0 | 10.5 | 8.7 | - |
| LLaMA 2-7B | 45.3 | - | 14.6 | 12.8 | 17.9 | - |
| Baichuan-7B | 42.3 | 42.8 | 9.7 | 9.2 | 26.6 | 44.4 |
| ChatGLM2-6B | 47.9 | 51.7 | 32.4 | 9.2 | - | 48.8 |
| InternLM-7B | 51.0 | 52.8 | 31.2 | 10.4 | 14.8 | - |
| Baichuan-13B | 51.6 | 53.6 | 26.6 | 12.8 | 30.0 | 55.8 |
| LLaMA-13B | 46.9 | 35.5 | 17.8 | 15.8 | 12.0 | - |
| LLaMA 2-13B | 54.8 | - | 28.7 | 18.3 | 24.2 | - |
| ChatGLM2-12B | 56.2 | **61.6** | 40.9 | - | - | - |
| **Qwen-7B** | **56.7** | 59.6 | **51.6** | **24.4** | **30.6** | **58.8** |
<p align="center">
<img src="assets/performance.png" width="1000"/>