GLM-130B is an open bilingual (English and Chinese) dense language model with 130 billion parameters, released by the Tsinghua KEG Lab and collaborators as part of the General Language Model (GLM) series. It is designed for large-scale inference and supports both left-to-right generation and blank filling, making it versatile across NLP tasks. Trained on over 400 billion tokens (200B English, 200B Chinese), it achieves performance surpassing GPT-3 175B, OPT-175B, and BLOOM-176B on multiple benchmarks, while also showing significant improvements on Chinese datasets compared to other large models. The model supports efficient inference via INT8 and INT4 quantization, reducing hardware requirements from 8× A100 GPUs to as little as a single server with 4× RTX 3090s. Built on the SwissArmyTransformer (SAT) framework and compatible with DeepSpeed and FasterTransformer, it supports high-speed inference (up to 2.5× faster) and reproducible evaluation across 30+ benchmark tasks.
Features
- 130B parameter bilingual dense model supporting English and Chinese
- Trained on 400B tokens (200B English + 200B Chinese)
- Outperforms GPT-3 175B, OPT-175B, and BLOOM-176B on multiple benchmarks
- Efficient inference with INT8/INT4 quantization on smaller hardware setups
- Compatible with SAT, DeepSpeed, Apex, and FasterTransformer for 2.5× speedup
- Open checkpoints and evaluation scripts for 30+ NLP tasks with reproducibility
Project Samples
