IP屬地：山東
-
LLaMA：開源的高效的基礎(chǔ)語(yǔ)言模型LLaMA: Open and Efficient Foundation Language Models Feb 2023 Hugo Touvr...
-
縮放律與模型架構(gòu)：歸納偏置如何影響縮放茄蚯？
Scaling Laws vs Model Architectures: How does Inductive Bias Influence S...
-
UL2：統(tǒng)一語(yǔ)言學(xué)習(xí)范式
UL2: Unifying Language Learning Paradigms https://arxiv.org/abs/2205.051...
-
用0.1%的額外計(jì)算超越縮放定律
Transcending Scaling Laws with 0.1% Extra Compute https://arxiv.org/abs/...
-
大語(yǔ)言模型的涌現(xiàn)能力
Emergent Abilities of Large Language Models https://arxiv.org/abs/2206.0...
-
預(yù)訓(xùn)練指南：測(cè)量數(shù)據(jù)年齡揭厚、領(lǐng)域覆蓋率、質(zhì)量和毒性的影響
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age...
-
自回歸生成建模的縮放律
Scaling Laws for Autoregressive Generative Modeling Oct 2020 https://arx...
-
神經(jīng)語(yǔ)言模型的縮放律
Scaling Laws for Neural Language Models Jan 2020 https://arxiv.org/abs/2...
-
DoReMi：優(yōu)化數(shù)據(jù)混合加速語(yǔ)言模型預(yù)訓(xùn)練
預(yù)訓(xùn)練數(shù)據(jù)域（如維基百科掀淘、書籍几颜、網(wǎng)絡(luò)文本）的混合比例極大地影響了語(yǔ)言模型（LM）的性能蚯撩。在本文中莫矗，我們提出了具有Minimax優(yōu)化的域重新加權(quán)（...