为什么Int4 的模型在EN上表现比BF16还要好?
#21
by
YijunYang280
- opened
效果评测 (Performance)
我们列出不同精度下模型在评测基准 TouchStone 上的表现，并发现量化模型并没有显著性能损失。结果如下所示：
We illustrate the model performance of both BF16 and Int4 models on the benchmark TouchStone, and we find that the quantized model does not suffer from significant performance degradation. Results are shown below:
| Quantization | ZH. | EN |
|---|---|---|
| BF16 | 401.2 | 645.2 |
| Int4 | 386.6 | 651.4 |
不太理解, 请提供一些insight