Falcon-40B: a state-of-the-art large LLM Apache 2.0 licensed

The Technology Innovation Institute, Abu Dhabi, just released a 40B parameters causal decoder-only LLM trained on 1,000B tokens of RefinedWeb.

Why is this noteworthy? They haven't shared the accompanying paper yet, so I will believe it when I see it, but they claim that it is the best open-source model currently available outperforming LLaMA, StableLM, RedPajama, MPT, etc. Fine. Progress.

But what stands out is that it is one of the only models made available under a permissive Apache 2.0 license allowing for commercial use, without any royalties or restrictions. Hopefully this will further spark competitive innovation from the open-source community.

For more information see the HuggingFace model card here.