GLM-130B

Open bilingual pre-trained model supporting Chinese and English

130B-parameter transformer-based large language model

Open repository with model/code licenses; permitted research and certain commercial uses per repo

Trained on 400B text tokens with the GLM training framework

Optimized large-scale training using the GLM library (parallelism and efficiency)

Strong reported performance across multiple NLP benchmarks

Inference and deployment scripts, including multi-GPU and mixed-precision

Downloadable checkpoints/weights for immediate use

Instruction-style and few-shot prompting capabilities

Associated ICLR 2023 paper describing model design and training