GitHub - openlm-research/open_llama: OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset
github.com
OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset - GitHub - openlm-research/open_llama: OpenLLaMA, a permissively licensed open sou...

cross-posted from: https://lemmy.blahaj.zone/post/74156

From the latest commits:

We are happy to release our final 1T token version of OpenLLaMA 3B and 7B. We’ve updated the evaluation results. We are also happy to release a 600B token preview of the 13B model, trained in collaboration with Stability AI.

Haven’t tried it yet, and the 13B model is still in the works, but hopefully this will be a better foundation than the leaked Meta AI model, not only for more reproducible research, but because nonacademics will be completely in the clear from a legal standpoint to run this stuff locally.