Models, Corpus, and Tools
We have released the models and tools developed by LLM-jp.
Some of the datasets and documentation in our development are planned to be released soon.
Open platforms
- Models: https://huggingface.co/llm-jp
- Tools: https://github.com/llm-jp
Major Models
Fine-tuned Models
Pre-trained Models
- LLM-jp-3-8x13b
- LLM-jp-3-172B (Access requires approval. Redistribution and certain uses are restricted.)
- LLM-jp-3-13B
Multi-model Models
Corpora for Pre-training
Evaluation and fine-tuning datasets
- ac-self-inst
- aya-ja-evol-inst
- Synthetic-JP-EN-Coding-Dataset
- wizardlm8x22b-logical-math-coding-sft-ja
- FLAN
- AnswerCarefully
- magpie-sft-v1.0
- databricks-dolly-15k-ja
- oasst1-21k-en
- oasst1-21k-ja
- oasst2-33k-ja
- oasst2-33k-en
- hh-rlhf-12k-ja
Other data is based on publicly available data, and details can be found in the “Evaluation Tools” and “Tuning Scripts” below, respectively.
Tools
- Pre-training Corpus Building Scripts v2.0
- Pre-training Corpus Building Scripts v1.0
- Tokenizer
- Evaluation Tools
- Fine-tuning Script
- trl-based
- Nemo-Aligner-based (Supports both SFT and DPO)
Leaderboards (Weights & Biases)
- llm-jp-eval
- Open LLM leaderboard（English benchmark) evaluation results of Japanese LLM
- VicunaQA Japanese benchmark
All Models
Fine-tuned Models
- LLM-jp-3
- LLM-jp-3-8x13b-instruct3
- LLM-jp-3-172B-instruct3
- LLM-jp-3-172B-beta2-instruct2 (Access requires approval. Redistribution and certain uses are restricted.)
- LLM-jp-3-172B-beta1-instruct (Ditto)
- LLM-jp-3-13B-instruct
- LLM-jp-3-8×1.8b-instruct3
- LLM-jp-3-7.2B-instruct3
- LLM-jp-3-3.7B-instruct3
- LLM-jp-3-1.8B-instruct3
- LLM-jp-3-980M-instruct3
- LLM-jp-3-440M-instruct3
- LLM-jp-3-150M-instruct3
- LLM-jp-3-8x13b-instruct2
- LLM-jp-3-172B-instruct2
- LLM-jp-3-13B-instruct2
- LLM-jp-3-8×1.8b-instruct2
- LLM-jp-3-7.2B-instruct2
- LLM-jp-3-3.7B-instruct2
- LLM-jp-3-1.8B-instruct2
- LLM-jp-3-980M-instruct2
- LLM-jp-3-440M-instruct2
- LLM-jp-3-150M-instruct2
- LLM-jp-3-13B-instruct
- LLM-jp-3-7.2B-instruct
- LLM-jp-3-3.7B-instruct
- LLM-jp-3-1.8B-instruct
- 13B v2.0
- 13B v1.1
- 13B v1.0
Pre-trained Models
- LLM-jp-3
- LLM-jp-3-8x13b
- LLM-jp-3-172B (Access requires approval. Redistribution and certain uses are restricted.)
- LLM-jp-3-172B-beta2 (Ditto)
- LLM-jp-3-172B-beta1 (Ditto)
- LLM-jp-3-13B
- LLM-jp-3-8×1.8b
- LLM-jp-3-7.2B
- LLM-jp-3-3.7B
- LLM-jp-3-1.8B
- LLM-jp-3-980M
- LLM-jp-3-440M
- LLM-jp-3-150M
- 13B v2.0
- 13B v1.0
- 1.3B v1.0