_toctree.yml 2.32 KB
Newer Older
1
2
3
4
5
- sections:
  - local: index
    title: Text Generation Inference
  - local: quicktour
    title: Quick Tour
6
7
  - local: supported_models
    title: Supported Models
fxmarty's avatar
fxmarty committed
8
9
10
11
12
13
14
15
  - local: installation_nvidia
    title: Using TGI with Nvidia GPUs
  - local: installation_amd
    title: Using TGI with AMD GPUs
  - local: installation_gaudi
    title: Using TGI with Intel Gaudi
  - local: installation_inferentia
    title: Using TGI with AWS Inferentia
Wang, Yi's avatar
Wang, Yi committed
16
17
  - local: installation_intel
    title: Using TGI with Intel GPUs
18
  - local: installation
fxmarty's avatar
fxmarty committed
19
    title: Installation from source
20

21
22
  - local: architecture
    title: Internal Architecture
23
24
  - local: usage_statistics
    title: Usage Statistics
25
26
27
28
29
30
31
32
  title: Getting started
- sections:
  - local: basic_tutorials/consuming_tgi
    title: Consuming TGI
  - local: basic_tutorials/preparing_model
    title: Preparing Model for Serving
  - local: basic_tutorials/gated_model_access
    title: Serving Private & Gated Models
Merve Noyan's avatar
Merve Noyan committed
33
34
  - local: basic_tutorials/using_cli
    title: Using TGI CLI
35
36
  - local: basic_tutorials/non_core_models
    title: Non-core Model Serving
37
38
  - local: basic_tutorials/safety
    title: Safety
39
40
  - local: basic_tutorials/using_guidance
    title: Using Guidance, JSON, tools
41
42
  - local: basic_tutorials/visual_language_models
    title: Visual Language Models
43
44
  - local: basic_tutorials/monitoring
    title: Monitoring TGI with Prometheus and Grafana
45
46
  - local: basic_tutorials/train_medusa
    title: Train Medusa
47
  title: Tutorials
48
49
50
51
52
53
54
55
- sections:
  - local: reference/launcher
    title: All TGI CLI options
  - local: reference/metrics
    title: Exported Metrics
  - local: reference/api_reference
    title: API Reference
  title: Reference
Omar Sanseviero's avatar
Omar Sanseviero committed
56
57
58
- sections:
  - local: conceptual/streaming
    title: Streaming
Merve Noyan's avatar
Merve Noyan committed
59
60
  - local: conceptual/quantization
    title: Quantization
61
62
  - local: conceptual/tensor_parallelism
    title: Tensor Parallelism
63
64
  - local: conceptual/paged_attention
    title: PagedAttention
65
66
  - local: conceptual/safetensors
    title: Safetensors
67
68
  - local: conceptual/flash_attention
    title: Flash Attention
Nicolas Patry's avatar
Nicolas Patry committed
69
70
71
  - local: conceptual/speculation
    title: Speculation (Medusa, ngram)
  - local: conceptual/guidance
72
    title: How Guidance Works (via outlines)
drbh's avatar
drbh committed
73
74
  - local: conceptual/lora
    title: LoRA (Low-Rank Adaptation)
75
76
  - local: conceptual/external
    title: External Resources
drbh's avatar
drbh committed
77

78

Omar Sanseviero's avatar
Omar Sanseviero committed
79
  title: Conceptual Guides