starlight-apk/Qwen
1
0
Fork 0
mirror of https://github.com/QwenLM/Qwen.git synced 2026-05-20 16:35:47 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update README_CN.md

Browse Source
This commit is contained in:
Yang An
2023-08-13 16:27:06 +08:00
committed by GitHub
parent 9863a4896f
commit 460ea3418b
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README_CN.md
View File

@@ -244,7 +244,7 @@ model = AutoModelForCausalLM.from_pretrained(
| Int8 | 52.8 | 10.44G | | Int8 | 52.8 | 10.44G |
| NF4 | 48.9 | 7.79G | | NF4 | 48.9 | 7.79G |
注：表中显存占用的测试环境为A100-SXM4-80G单卡，PyTorch 2.0.1，cuda11.8，开启flash attention 注：表中显存占用的测试环境为A100-SXM4-80G单卡，PyTorch 2.0.1，CUDA 11.8，开启flash attention
## 推理性能 ## 推理性能
@@ -258,7 +258,7 @@ model = AutoModelForCausalLM.from_pretrained(
| Int8 (bnb) | 7.94 | 7.86 | | Int8 (bnb) | 7.94 | 7.86 |
| NF4 (bnb) | 21.43 | 20.37 | | NF4 (bnb) | 21.43 | 20.37 |
具体的评测方式为：指定输入context长度为1，生成长度为2048；测试硬件为A100-SXM4-80G单卡，软件环境为PyTorch 2.0.1，cuda版本11.8，计算生成该2048序列的平均速度 具体的评测方式为：指定输入context长度为1，生成长度为2048；测试硬件为A100-SXM4-80G单卡，软件环境为PyTorch 2.0.1，CUDA版本11.8，计算生成该2048序列的平均速度
### 显存占用 ### 显存占用