Hugging Face's logo

internlm
/
internlm3-8b-instruct

Text Generation
Safetensors
internlm3
conversational
custom_code
Model card Files
xet
Community
21
haijunlv commited on
Commit
011ff6d
·
verified ·
1 Parent(s): dc29aeb

Upload README.md (#15)

Browse files

- Upload README.md (68da3cc661ec5983abe2d68b7d24e2adb09f40b0)

Files changed (1) hide show
  1. README.md +40 -38
README.md CHANGED
@@ -48,25 +48,26 @@ InternLM3 supports both the deep thinking mode for solving complicated reasoning
48
 
49
  We conducted a comprehensive evaluation of InternLM using the open-source evaluation tool [OpenCompass](https://github.com/internLM/OpenCompass/). The evaluation covered five dimensions of capabilities: disciplinary competence, language competence, knowledge competence, inference competence, and comprehension competence. Here are some of the evaluation results, and you can visit the [OpenCompass leaderboard](https://rank.opencompass.org.cn) for more evaluation results.
50
 
51
- | Benchmark | | InternLM3-8B-Instruct | Qwen2.5-7B-Instruct | Llama3.1-8B-Instruct | GPT-4o-mini(close source) |
52
- | ------------ | ------------------------------- | --------------------- | ------------------- | -------------------- | ------------------------- |
53
- | General | CMMLU(0-shot) | **83.1** | 75.8 | 53.9 | 66.0 |
54
- | | MMLU(0-shot) | 76.6 | **76.8** | 71.8 | 82.7 |
55
- | | MMLU-Pro(0-shot) | **57.6** | 56.2 | 48.1 | 64.1 |
56
- | Reasoning | GPQA-Diamond(0-shot) | **37.4** | 33.3 | 24.2 | 42.9 |
57
- | | DROP(0-shot) | **83.1** | 80.4 | 81.6 | 85.2 |
58
- | | HellaSwag(10-shot) | **91.2** | 85.3 | 76.7 | 89.5 |
59
- | | KOR-Bench(0-shot) | **56.4** | 44.6 | 47.7 | 58.2 |
60
- | MATH | MATH-500(0-shot) | **83.0*** | 72.4 | 48.4 | 74.0 |
61
- | | AIME2024(0-shot) | **20.0*** | 16.7 | 6.7 | 13.3 |
62
- | Coding | LiveCodeBench(2407-2409 Pass@1) | **17.8** | 16.8 | 12.9 | 21.8 |
63
- | | HumanEval(Pass@1) | 82.3 | **85.4** | 72.0 | 86.6 |
64
- | Instrunction | IFEval(Prompt-Strict) | **79.3** | 71.7 | 75.2 | 79.7 |
65
- | Long Context | RULER(4-128K Average) | 87.9 | 81.4 | **88.5** | 90.7 |
66
- | Chat | AlpacaEval 2.0(LC WinRate) | **51.1** | 30.3 | 25.0 | 50.7 |
67
- | | WildBench(Raw Score) | **33.1** | 23.3 | 1.5 | 40.3 |
68
- | | MT-Bench-101(Score 1-10) | **8.59** | 8.49 | 8.37 | 8.87 |
69
-
 
70
  - The evaluation results were obtained from [OpenCompass](https://github.com/internLM/OpenCompass/) (some data marked with *, which means evaluating with Thinking Mode), and evaluation configuration can be found in the configuration files provided by [OpenCompass](https://github.com/internLM/OpenCompass/).
71
  - The evaluation data may have numerical differences due to the version iteration of [OpenCompass](https://github.com/internLM/OpenCompass/), so please refer to the latest evaluation results of [OpenCompass](https://github.com/internLM/OpenCompass/).
72
 
@@ -470,25 +471,26 @@ InternLM3支持通过长思维链求解复杂推理任务的深度思考模式
470
 
471
  我们使用开源评测工具 [OpenCompass](https://github.com/internLM/OpenCompass/) 从学科综合能力、语言能力、知识能力、推理能力、理解能力五大能力维度对InternLM开展全面评测，部分评测结果如下表所示，欢迎访问[ OpenCompass 榜单 ](https://rank.opencompass.org.cn)获取更多的评测结果。
472
 
473
- | 评测集\模型 | | InternLM3-8B-Instruct | Qwen2.5-7B-Instruct | Llama3.1-8B-Instruct | GPT-4o-mini(close source) |
474
- | ------------ | ------------------------------- | --------------------- | ------------------- | -------------------- | ------------------------- |
475
- | General | CMMLU(0-shot) | **83.1** | 75.8 | 53.9 | 66.0 |
476
- | | MMLU(0-shot) | 76.6 | **76.8** | 71.8 | 82.7 |
477
- | | MMLU-Pro(0-shot) | **57.6** | 56.2 | 48.1 | 64.1 |
478
- | Reasoning | GPQA-Diamond(0-shot) | **37.4** | 33.3 | 24.2 | 42.9 |
479
- | | DROP(0-shot) | **83.1** | 80.4 | 81.6 | 85.2 |
480
- | | HellaSwag(10-shot) | **91.2** | 85.3 | 76.7 | 89.5 |
481
- | | KOR-Bench(0-shot) | **56.4** | 44.6 | 47.7 | 58.2 |
482
- | MATH | MATH-500(0-shot) | **83.0*** | 72.4 | 48.4 | 74.0 |
483
- | | AIME2024(0-shot) | **20.0*** | 16.7 | 6.7 | 13.3 |
484
- | Coding | LiveCodeBench(2407-2409 Pass@1) | **17.8** | 16.8 | 12.9 | 21.8 |
485
- | | HumanEval(Pass@1) | 82.3 | **85.4** | 72.0 | 86.6 |
486
- | Instrunction | IFEval(Prompt-Strict) | **79.3** | 71.7 | 75.2 | 79.7 |
487
- | LongContext | RULER(4-128K Average) | 87.9 | 81.4 | **88.5** | 90.7 |
488
- | Chat | AlpacaEval 2.0(LC WinRate) | **51.1** | 30.3 | 25.0 | 50.7 |
489
- | | WildBench(Raw Score) | **33.1** | 23.3 | 1.5 | 40.3 |
490
- | | MT-Bench-101(Score 1-10) | **8.59** | 8.49 | 8.37 | 8.87 |
491
-
 
492
  - 以上评测结果基于 [OpenCompass](https://github.com/internLM/OpenCompass/) 获得(部分数据标注`*`代表使用深度思考模式进行评测)，具体测试细节可参见 [OpenCompass](https://github.com/internLM/OpenCompass/) 中提供的配置文件。
493
  - 评测数据会因 [OpenCompass](https://github.com/internLM/OpenCompass/) 的版本迭代而存在数值差异，请以 [OpenCompass](https://github.com/internLM/OpenCompass/) 最新版的评测结果为主。
494
 
 
48
 
49
  We conducted a comprehensive evaluation of InternLM using the open-source evaluation tool [OpenCompass](https://github.com/internLM/OpenCompass/). The evaluation covered five dimensions of capabilities: disciplinary competence, language competence, knowledge competence, inference competence, and comprehension competence. Here are some of the evaluation results, and you can visit the [OpenCompass leaderboard](https://rank.opencompass.org.cn) for more evaluation results.
50
 
51
+ | | Benchmark | InternLM3-8B-Instruct | Qwen2.5-7B-Instruct | Llama3.1-8B-Instruct | GPT-4o-mini(closed source) |
52
+ | ------------ | ------------------------------- | --------------------- | ------------------- | -------------------- | -------------------------- |
53
+ | General | CMMLU(0-shot) | **83.1** | 75.8 | 53.9 | 66.0 |
54
+ | | MMLU(0-shot) | 76.6 | **76.8** | 71.8 | 82.7 |
55
+ | | MMLU-Pro(0-shot) | **57.6** | 56.2 | 48.1 | 64.1 |
56
+ | Reasoning | GPQA-Diamond(0-shot) | **37.4** | 33.3 | 24.2 | 42.9 |
57
+ | | DROP(0-shot) | **83.1** | 80.4 | 81.6 | 85.2 |
58
+ | | HellaSwag(10-shot) | **91.2** | 85.3 | 76.7 | 89.5 |
59
+ | | KOR-Bench(0-shot) | **56.4** | 44.6 | 47.7 | 58.2 |
60
+ | MATH | MATH-500(0-shot) | **83.0*** | 72.4 | 48.4 | 74.0 |
61
+ | | AIME2024(0-shot) | **20.0*** | 16.7 | 6.7 | 13.3 |
62
+ | Coding | LiveCodeBench(2407-2409 Pass@1) | **17.8** | 16.8 | 12.9 | 21.8 |
63
+ | | HumanEval(Pass@1) | 82.3 | **85.4** | 72.0 | 86.6 |
64
+ | Instrunction | IFEval(Prompt-Strict) | **79.3** | 71.7 | 75.2 | 79.7 |
65
+ | Long Context | RULER(4-128K Average) | 87.9 | 81.4 | **88.5** | 90.7 |
66
+ | Chat | AlpacaEval 2.0(LC WinRate) | **51.1** | 30.3 | 25.0 | 50.7 |
67
+ | | WildBench(Raw Score) | **33.1** | 23.3 | 1.5 | 40.3 |
68
+ | | MT-Bench-101(Score 1-10) | **8.59** | 8.49 | 8.37 | 8.87 |
69
+
70
+ - Values marked in bold indicate the **highest** in open source models
71
  - The evaluation results were obtained from [OpenCompass](https://github.com/internLM/OpenCompass/) (some data marked with *, which means evaluating with Thinking Mode), and evaluation configuration can be found in the configuration files provided by [OpenCompass](https://github.com/internLM/OpenCompass/).
72
  - The evaluation data may have numerical differences due to the version iteration of [OpenCompass](https://github.com/internLM/OpenCompass/), so please refer to the latest evaluation results of [OpenCompass](https://github.com/internLM/OpenCompass/).
73
 
 
471
 
472
  我们使用开源评测工具 [OpenCompass](https://github.com/internLM/OpenCompass/) 从学科综合能力、语言能力、知识能力、推理能力、理解能力五大能力维度对InternLM开展全面评测，部分评测结果如下表所示，欢迎访问[ OpenCompass 榜单 ](https://rank.opencompass.org.cn)获取更多的评测结果。
473
 
474
+ | | 评测集\模型 | InternLM3-8B-Instruct | Qwen2.5-7B-Instruct | Llama3.1-8B-Instruct | GPT-4o-mini(闭源) |
475
+ | ------------ | ------------------------------- | --------------------- | ------------------- | -------------------- | ----------------- |
476
+ | General | CMMLU(0-shot) | **83.1** | 75.8 | 53.9 | 66.0 |
477
+ | | MMLU(0-shot) | 76.6 | **76.8** | 71.8 | 82.7 |
478
+ | | MMLU-Pro(0-shot) | **57.6** | 56.2 | 48.1 | 64.1 |
479
+ | Reasoning | GPQA-Diamond(0-shot) | **37.4** | 33.3 | 24.2 | 42.9 |
480
+ | | DROP(0-shot) | **83.1** | 80.4 | 81.6 | 85.2 |
481
+ | | HellaSwag(10-shot) | **91.2** | 85.3 | 76.7 | 89.5 |
482
+ | | KOR-Bench(0-shot) | **56.4** | 44.6 | 47.7 | 58.2 |
483
+ | MATH | MATH-500(0-shot) | **83.0*** | 72.4 | 48.4 | 74.0 |
484
+ | | AIME2024(0-shot) | **20.0*** | 16.7 | 6.7 | 13.3 |
485
+ | Coding | LiveCodeBench(2407-2409 Pass@1) | **17.8** | 16.8 | 12.9 | 21.8 |
486
+ | | HumanEval(Pass@1) | 82.3 | **85.4** | 72.0 | 86.6 |
487
+ | Instrunction | IFEval(Prompt-Strict) | **79.3** | 71.7 | 75.2 | 79.7 |
488
+ | LongContext | RULER(4-128K Average) | 87.9 | 81.4 | **88.5** | 90.7 |
489
+ | Chat | AlpacaEval 2.0(LC WinRate) | **51.1** | 30.3 | 25.0 | 50.7 |
490
+ | | WildBench(Raw Score) | **33.1** | 23.3 | 1.5 | 40.3 |
491
+ | | MT-Bench-101(Score 1-10) | **8.59** | 8.49 | 8.37 | 8.87 |
492
+
493
+ - 表中标粗的数值表示在对比的开源模型中的最高值。
494
  - 以上评测结果基于 [OpenCompass](https://github.com/internLM/OpenCompass/) 获得(部分数据标注`*`代表使用深度思考模式进行评测)，具体测试细节可参见 [OpenCompass](https://github.com/internLM/OpenCompass/) 中提供的配置文件。
495
  - 评测数据会因 [OpenCompass](https://github.com/internLM/OpenCompass/) 的版本迭代而存在数值差异，请以 [OpenCompass](https://github.com/internLM/OpenCompass/) 最新版的评测结果为主。
496