"megatron/training/tokenizer/bert_tokenization.py" did not exist on "17a6044dedf7b6e8573338e7201097f4dd837e5b"
README.md 1.07 KB
Newer Older
yuguo960516's avatar
bloom  
yuguo960516 committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
## Introduction
Here are projects built based on LiBai, which show you how to use LiBai as a library, to make your own projects more maintainable.

## Projects on LiBai
- [BLOOM: BigScience Large Open-science Open-access Multilingual Language Model](./BLOOM/)
- [CLIP: Contrastive Language-Image Pre-Training](./CLIP)
- [Couplets](./Couplets/)
- [DALLE2](./DALLE2/)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](./GLM/)
- [MAE: Masked Autoencoders Are Scalable Vision Learners](./MAE)
- [MagicPrompt-Stable-Diffusion](./MagicPrompt/)
- [MOCOV3: An Empirical Study of Training Self-Supervised Vision Transformers](./MOCOV3)
- [MT5](./MT5/)
- [NeRF](./NeRF/)
- [PaLM: Pathways Language Model](./PaLM)
- [QQP: Quora Question Pairs](./QQP)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](./SimCSE)
- [T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](./T5)
- [Text Classification](./text_classification)
- [mock_transformers: Distributed inference in LiBai with model under Transformers](./mock_transformers/)