LLaMA: Open and Efficient Foundation Language Models Feb 2023 Hugo Touvr...
使用检查点支持容错训练 在整个RLHF训练过程中,可能会出现训练错误或机器故障,因此建议启用检查点功能以最小化损失。 API接口已在 :ref:...
Scaling Laws vs Model Architectures: How does Inductive Bias Influence S...
UL2: Unifying Language Learning Paradigms https://arxiv.org/abs/2205.051...
Transcending Scaling Laws with 0.1% Extra Compute https://arxiv.org/abs/...
Emergent Abilities of Large Language Models https://arxiv.org/abs/2206.0...
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age...
Scaling Laws for Autoregressive Generative Modeling Oct 2020 https://arx...
Scaling Laws for Neural Language Models Jan 2020 https://arxiv.org/abs/2...