Open-Source Large Language Models (LLMs)
- 01. GPT-NeoX leverages many of the same features and technologies as the popular Megatron-DeepSpeed library but with substantially increased usability and novel optimizations.
- 02. Alpa is a system for training and serving gigantic machine learning models. Alpa makes training and serving large models like GPT-3 simple, affordable, accessible to everyone.
- 03. BLOOM is an autoregressive LLM trained to continue text from a prompt on vast amounts of text data using industrial-scale computational resources
- 04. GPT-J 6B is a transformer model trained using Ben Wang's Mesh Transformer JAX. "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
- 05. GLM-130B is an open bilingual (English & Chinese) bidirectional dense model with 130 billion parameters, pre-trained using the algorithm of General Language Model (GLM).
- 06. Cerebras open sources seven GPT-3 models from 111 million to 13 billion parameters. Trained using the Chinchilla formula, these models set new benchmarks for accuracy and compute efficiency.
- 07. FLAN-T5 model is a encoder-decoder model that has been pre-trained on a multi-task mixture of unsupervised and supervised tasks and for which each task is converted into a text-to-text format.
- 08. Llama 2 - The next generation of our open source large language model. Llama 2 is available for free for research and commercial use.
- 09. Alpaca 7B, a model fine-tuned from the LLaMA 7B model on 52K instruction-following demonstrations. On evaluation of single-turn instruction following, Alpaca behaves qualitatively similarly to OpenAI’s text-davinci-003.
- 10. Falcon 180B is a super-powerful language model with 180 billion parameters, trained on 3.5 trillion tokens.
- 11. Code Llama, a LLM that can use text prompts to generate and discuss code. Code Llama is state-of-the-art for publicly available LLMs on coding tasks.
- 12. Mistral-7B-v0.1 is a small, yet powerful model adaptable to many use-cases. Mistral 7B is better than Llama 2 13B on all benchmarks, has natural coding abilities, and 8k sequence length.
- 13. Vicuna v1.3 is fine-tuned from LLaMA with supervised instruction fine-tuning. The training data is around 125K conversations collected from ShareGPT.com.
- 14. Claude 2 is a friendly, enthusiastic colleague or personal assistant who can be instructed in natural language to help you with many tasks.
- 15. Open Pre-trained Transformers (OPT), a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters. OPT-175B is comparable to GPT-3, while requiring only 1/7th the carbon footprint to develop.
- 16. Baichuan-7B is an open-source large-scale pre-trained mode with 7 billion parameters trained on approximately 1.2 trillion tokens. It supports both Chinese and English, with a context window length of 4096.
- 17. CodeGen is a family of open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.
- 18. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available for commercial use,
- 19. DALL·E 2 is an AI system that can create realistic images and art from a description in natural language.