Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
libai.models.rst 1.68 KB
Newer Older
yuguo960516's avatar
gpt2  
yuguo960516 committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83 
libai.models
##############################
Supported models in LiBai(李白)

- `VisionTransformer`_
- `SwinTransformer`_
- `SwinTransformerV2`_
- `ResMLP`_
- `BERT`_
- `RoBERTa`_
- `T5`_
- `GPT-2`_

.. _VisionTransformer: https://arxiv.org/abs/2010.11929
.. _SwinTransformer: https://arxiv.org/abs/2103.14030
.. _SwinTransformerV2: https://arxiv.org/abs/2111.09883
.. _ResMLP: https://arxiv.org/abs/2105.03404
.. _BERT: https://arxiv.org/abs/1810.04805
.. _RoBERTa: https://arxiv.org/abs/1907.11692
.. _T5: https://arxiv.org/abs/1910.10683
.. _GPT-2: https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf



.. currentmodule:: libai.models

.. automodule:: libai.models.build
    :members:
        build_model,
        build_graph,

VisionTransformer
-----------------
.. automodule:: libai.models.vision_transformer
    :members: 
        VisionTransformer,

SwinTransformer
---------------
.. automodule:: libai.models.swin_transformer
    :members: 
        SwinTransformer,

SwinTransformerV2
-----------------
.. automodule:: libai.models.swin_transformer_v2
    :members:
        SwinTransformerV2,

ResMLP
-------
.. automodule:: libai.models.resmlp
    :members:
        ResMLP,

BERT
----
.. automodule:: libai.models.bert_model
    :members: 
        BertModel,
        BertForPreTraining,

RoBERTa
-------
.. automodule:: libai.models.roberta_model
    :members: 
        RobertaModel,
        RobertaForPreTraining,
        RobertaForCausalLM,

T5
---
.. automodule:: libai.models.t5_model
    :members: 
        T5ForPreTraining,
        T5Model,

GPT-2
-----
.. automodule:: libai.models.gpt_model
    :members: 
        GPTForPreTraining,
        GPTModel