Commit 48c080c4 authored by songlinfeng's avatar songlinfeng
Browse files

rename dcu-ctk dcu-container-toolkit

parent 0ccfece1
......@@ -4,7 +4,7 @@ DIST_DIR ?= $(CURDIR)/dist
include $(CURDIR)/versions.mk
MODULE := dtk-container-toolkit
MODULE := dcu-container-toolkit
# By default run all native docker-based targets
docker-native:
......@@ -29,7 +29,7 @@ CLI_VERSION = $(LIB_VERSION)$(if $(LIB_TAG),-$(LIB_TAG))
else
CLI_VERSION = $(VERSION)
endif
CLI_VERSION_PACKAGE = dtk-container-toolkit/internal/info
CLI_VERSION_PACKAGE = dcu-container-toolkit/internal/info
binaries: cmds
ifneq ($(PREFIX),)
......
......@@ -4,8 +4,8 @@
DTK Container Toolkit 使用户能够构建和运行使用DCU设备的容器,该toolkit包括以下工具包。
- ```dtk-container-toolkit``` - DTK容器运行时
- ```dtk-ctk``` - DTK容器工具集命令行
- ```dcu-container-toolkit``` - DTK容器运行时
- ```dcu-ctk``` - DTK容器工具集命令行
## 使用
- --gpus需要Docker version 19+
......@@ -16,8 +16,8 @@ DTK Container Toolkit 使用户能够构建和运行使用DCU设备的容器,该
使用 dpkg/rpm -i 进行安装。安装后会自动执行以下命令
```sh
$ dtk-ctk --quiet config --config-file=/etc/dtk-container-runtime/config.toml --in-place #生成配置文件
$ dtk-ctk runtime configure --runtime=docker --set-as-default #修改docker的config.json的runtime
$ dcu-ctk --quiet config --config-file=/etc/dcu-container-runtime/config.toml --in-place #生成配置文件
$ dcu-ctk runtime configure --runtime=docker --set-as-default #修改docker的config.json的runtime
```
重启docker服务
......@@ -35,21 +35,21 @@ $ docker run -it --gpus 1 ubuntu:18.04 # 添加一个HCU设备,HCU 0
$ docker run -it --gpus "device=0,2" ubuntu:18.04 #添加第0号和第2号GPU
```
#### 通过环境变量 `DTK_VISIBLE_DEVICES`
可以通过 docker run 添加环境变量 -e DTK_VISIBLE_DEVICES 给容器添加HCU设备。
#### 通过环境变量 `DCU_VISIBLE_DEVICES`
可以通过 docker run 添加环境变量 -e DCU_VISIBLE_DEVICES 给容器添加HCU设备。
```sh
docker run -it -e DTK_VISIBLE_DEVICES=all ubuntu:18.04 # 添加所有HCU设备
docker run -it -e DTK_VISIBLE_DEVICES=0 ubuntu:18.04 # 添加HCU设备0
docker run -it -e DTK_VISIBLE_DEVICES=0,1 ubuntu:18.04 # 添加HCU设备0、1
docker run -it -e DCU_VISIBLE_DEVICES=all ubuntu:18.04 # 添加所有HCU设备
docker run -it -e DCU_VISIBLE_DEVICES=0 ubuntu:18.04 # 添加HCU设备0
docker run -it -e DCU_VISIBLE_DEVICES=0,1 ubuntu:18.04 # 添加HCU设备0、1
```
#### 通过CDI方式
- 首先,生成CDI spec文件
```sh
$ dtk-ctk cdi generate --output=/etc/cdi/dtk.json
$ dcu-ctk cdi generate --output=/etc/cdi/dtk.json
```
- 配置docker开启CDI
```sh
$ dtk-ctk runtime configure --runtime=docker --set-as-default --cdi.enabled
$ dcu-ctk runtime configure --runtime=docker --set-as-default --cdi.enabled
```
- 使用所有的DCU
```sh
......@@ -63,15 +63,15 @@ $ docker run --rm --device c-3000.com/hcu=0 --device c-3000.com/hcu=1 -it ubuntu
podman 需要version 2.0+
- 首先需要修改podman的runtime
```sh
$ dtk-ctk runtime configure --runtime=podman --set-as-default
$ dcu-ctk runtime configure --runtime=podman --set-as-default
```
- 通过环境变量方式使用
```sh
$ podman run -it -e DTK_VISIBLE_DEVICES=all ubuntu:18.04
$ podman run -it -e DCU_VISIBLE_DEVICES=all ubuntu:18.04
```
### 列出可使用的DCU
```sh
$ dtk-ctk cdi list
$ dcu-ctk cdi list
INFO[0000] Found 3 CDI devices
c-3000.com/hcu=0
c-3000.com/hcu=1
......@@ -84,34 +84,34 @@ c-3000.com/hcu=hcu-73873c7a6eb040a1
### docker rootless下对文件读写权限的限制
非root用户在使用-v挂载目录时,保留原有权限,无法对ro目录添加w权限,执行该命令需要root权限
```sh
$ dtk-ctk rootless --runtime=docker
$ dcu-ctk rootless --runtime=docker
```
### DCU Tracker
DCU Tracker用来监控使用--gpus和-e DTK_VISIBLE_DEVICES启动容器的DCU使用情况,也可以设置DCU被独享或共享。默认情况下DCU Tracker是disable状态,用户可以使用dtk-ctk的命令行来enable.
DCU Tracker用来监控使用--gpus和-e DCU_VISIBLE_DEVICES启动容器的DCU使用情况,也可以设置DCU被独享或共享。默认情况下DCU Tracker是disable状态,用户可以使用dcu-ctk的命令行来enable.
DCU Tracker 提供命令行来控制容器对DCU的访问,可以被设置为shared或exclusive.
- shared 表示DCU可以同时被多个容器一起使用,这是默认状态
- exclusive 表示DCU同时只能被一个容器使用。
```sh
$ dtk-ctk dcu-tracker -h
$ dcu-ctk dcu-tracker -h
NAME:
C-3000 DTK Container Toolkit CLI dcu-tracker - DCU Tracker related commands
USAGE:
dtk-ctk dcu-tracker [dcu-ids] [accessibility]
dcu-ctk dcu-tracker [dcu-ids] [accessibility]
Arguments:
dcu-ids Comma-separated list of DCU IDs (comma separated list, range operator, all)
accessibility Must be either 'exclusive' or 'shared'
Examples:
dtk-ctk dcu-tracker 0,1,2 exclusive
dtk-ctk dcu-tracker 0,1-2 shared
dtk-ctk dcu-tracker all shared
dcu-ctk dcu-tracker 0,1,2 exclusive
dcu-ctk dcu-tracker 0,1-2 shared
dcu-ctk dcu-tracker all shared
OR
dtk-ctk dcu-tracker [command] [options]
dcu-ctk dcu-tracker [command] [options]
COMMANDS:
disable Disable the DCU Tracker
......@@ -146,7 +146,7 @@ GPU Temp (DieEdge) AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU%
如果DCU Tracker enabled,DCU默认被赋予 shared 权限
```sh
$ dtk-ctk dcu-tracker status
$ dcu-ctk dcu-tracker status
------------------------------------------------------------------------------------------------------------------------
GPU Id UUID Accessibility Container Ids
------------------------------------------------------------------------------------------------------------------------
......@@ -157,7 +157,7 @@ GPU Temp (DieEdge) AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU%
如果DCU Tracker没有开启,则会有相应提示
```sh
$ dtk-ctk dcu-tracker status
$ dcu-ctk dcu-tracker status
DCU Tracker is disabled
```
......@@ -165,20 +165,20 @@ GPU Temp (DieEdge) AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU%
```sh
$ dtk-ctk dcu-tracker status
$ dcu-ctk dcu-tracker status
DCU Tracker is disabled
$ dtk-ctk dcu-tracker enable
$ dcu-ctk dcu-tracker enable
DCU Tracker has been enabled
$ dtk-ctk dcu-tracker enable
$ dcu-ctk dcu-tracker enable
DCU Tracker is already enabled
```
- 关闭 DCU Tracker
```sh
$ dtk-ctk dcu-tracker disable
$ dcu-ctk dcu-tracker disable
DCU Tracker has been disabled
```
......@@ -186,11 +186,11 @@ GPU Temp (DieEdge) AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU%
当DCU Tracker开启时,启动容器时会自动记录容器使用DCU的情况
```sh
$ docker run --name slf_dmps -e DTK_VISIBLE_DEVICES=0,1 -it a4dd5be0ca23
$ docker run --name slf_dmps -e DCU_VISIBLE_DEVICES=0,1 -it a4dd5be0ca23
$ docker run --name slf_dmp -e DTK_VISIBLE_DEVICES=0,1 -it a4dd5be0ca23
$ docker run --name slf_dmp -e DCU_VISIBLE_DEVICES=0,1 -it a4dd5be0ca23
$ dtk-ctk dcu-tracker status
$ dcu-ctk dcu-tracker status
------------------------------------------------------------------------------------------------------------------------
GPU Id UUID Accessibility Container Ids
------------------------------------------------------------------------------------------------------------------------
......@@ -202,7 +202,7 @@ GPU Temp (DieEdge) AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU%
$ docker rm slf_dmp
$ dtk-ctk dcu-tracker status
$ dcu-ctk dcu-tracker status
------------------------------------------------------------------------------------------------------------------------
GPU Id UUID Accessibility Container Ids
------------------------------------------------------------------------------------------------------------------------
......@@ -215,10 +215,10 @@ GPU Temp (DieEdge) AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU%
- 设置DCU 为exclusive属性
```sh
$ dtk-ctk dcu-tracker 1 exclusive
$ dcu-ctk dcu-tracker 1 exclusive
DCUs [1] have been made exclusive
$ dtk-ctk dcu-tracker status
$ dcu-ctk dcu-tracker status
------------------------------------------------------------------------------------------------------------------------
GPU Id UUID Accessibility Container Ids
------------------------------------------------------------------------------------------------------------------------
......@@ -226,7 +226,7 @@ GPU Temp (DieEdge) AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU%
1 0x73873C7A6EB008A1 Exclusive dc3c3153ab2e1cde5013a5e5d116cf467894949d1ef4b29ba8caa23a40f66d8d
2 0x73873C7A6EB040A1 Shared None
$ docker run --name slf_dmp -e DTK_VISIBLE_DEVICES=0,1 -it a4dd5be0ca23
$ docker run --name slf_dmp -e DCU_VISIBLE_DEVICES=0,1 -it a4dd5be0ca23
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: failed to
create DTK Container Runtime: failed to construct OCI spec modifier: failed to reserve DCUs: DCUs [1] are exclusive and already in use: unknown.
......
......@@ -5,8 +5,8 @@
package chmod
import (
"dtk-container-toolkit/internal/logger"
"dtk-container-toolkit/internal/oci"
"dcu-container-toolkit/internal/logger"
"dcu-container-toolkit/internal/oci"
"errors"
"fmt"
"io/fs"
......
......@@ -5,15 +5,15 @@
package commands
import (
"dtk-container-toolkit/cmd/dtk-cdi-hook/chmod"
symlinks "dtk-container-toolkit/cmd/dtk-cdi-hook/create-symlinks"
"dtk-container-toolkit/internal/logger"
"dcu-container-toolkit/cmd/dcu-cdi-hook/chmod"
symlinks "dcu-container-toolkit/cmd/dcu-cdi-hook/create-symlinks"
"dcu-container-toolkit/internal/logger"
"github.com/urfave/cli/v2"
)
// New creates the commands associated with supported CDI hooks.
// These are shared by the dtk-cdi-hook and dtk-ctk hook commands.
// These are shared by the dcu-cdi-hook and dcu-ctk hook commands.
func New(logger logger.Interface) []*cli.Command {
return []*cli.Command{
symlinks.NewCommand(logger),
......
......@@ -5,8 +5,8 @@
package symlinks
import (
"dtk-container-toolkit/internal/logger"
"dtk-container-toolkit/internal/oci"
"dcu-container-toolkit/internal/logger"
"dcu-container-toolkit/internal/oci"
"fmt"
"os"
"path/filepath"
......
......@@ -5,8 +5,8 @@
package main
import (
"dtk-container-toolkit/cmd/dtk-cdi-hook/commands"
"dtk-container-toolkit/internal/info"
"dcu-container-toolkit/cmd/dcu-cdi-hook/commands"
"dcu-container-toolkit/internal/info"
"os"
"github.com/sirupsen/logrus"
......
# DTK Container Runtime
DTK Container Runtime 是一个专为符合 OCI (Open Container Initiative,开放容器计划)规范的底层运行时( 如[runc](https://github.com/opencontainers/runc) )设计的垫片(shim)。当 dtk-container-runtime 接收到一个 `create` 命令时,它会对传入的 [OCI运行时规范](https://github.com/opencontainers/runtime-spec) 进行即时修改,以添加对 HCU 的特殊支持,然后将这个修改后的命令转发给底层的运行时(如runc、containerd等)。
DTK Container Runtime 是一个专为符合 OCI (Open Container Initiative,开放容器计划)规范的底层运行时( 如[runc](https://github.com/opencontainers/runc) )设计的垫片(shim)。当 dcu-container-runtime 接收到一个 `create` 命令时,它会对传入的 [OCI运行时规范](https://github.com/opencontainers/runtime-spec) 进行即时修改,以添加对 HCU 的特殊支持,然后将这个修改后的命令转发给底层的运行时(如runc、containerd等)。
## 配置
DTK Container Runtime (DCR) 使用基于文件的配置方式,其配置文件存储在 /etc/dtk-container-runtime/config.toml 路径下。安装 dtk-container-toolkit 时会自动生成该文件。
DTK Container Runtime (DCR) 使用基于文件的配置方式,其配置文件存储在 /etc/dcu-container-runtime/config.toml 路径下。安装 dcu-container-toolkit 时会自动生成该文件。
......@@ -5,8 +5,8 @@
package main
import (
"dtk-container-toolkit/internal/dcu-tracker"
"dtk-container-toolkit/internal/runtime"
"dcu-container-toolkit/internal/dcu-tracker"
"dcu-container-toolkit/internal/runtime"
"os"
//"log"
)
......
......@@ -5,10 +5,10 @@
package cdi
import (
"dtk-container-toolkit/cmd/dtk-ctk/cdi/generate"
"dtk-container-toolkit/cmd/dtk-ctk/cdi/list"
"dtk-container-toolkit/cmd/dtk-ctk/cdi/transform"
"dtk-container-toolkit/internal/logger"
"dcu-container-toolkit/cmd/dcu-ctk/cdi/generate"
"dcu-container-toolkit/cmd/dcu-ctk/cdi/list"
"dcu-container-toolkit/cmd/dcu-ctk/cdi/transform"
"dcu-container-toolkit/internal/logger"
"github.com/urfave/cli/v2"
)
......
......@@ -5,11 +5,11 @@
package generate
import (
"dtk-container-toolkit/internal/config"
"dtk-container-toolkit/internal/logger"
"dtk-container-toolkit/pkg/c3000cdi"
"dtk-container-toolkit/pkg/c3000cdi/spec"
"dtk-container-toolkit/pkg/c3000cdi/transform"
"dcu-container-toolkit/internal/config"
"dcu-container-toolkit/internal/logger"
"dcu-container-toolkit/pkg/c3000cdi"
"dcu-container-toolkit/pkg/c3000cdi/spec"
"dcu-container-toolkit/pkg/c3000cdi/transform"
"fmt"
"os"
"path/filepath"
......@@ -110,11 +110,11 @@ func (m command) build() *cli.Command {
Destination: &opts.librarySearchPaths,
},
&cli.StringFlag{
Name: "dtk-cdi-hook-path",
Aliases: []string{"dtk-ctk-path"},
Usage: "Specify the path to use for the dtk-cdi-hook in the generated CDI specification. " +
"If not specified, the PATH will be searched for `dtk-cdi-hook`. " +
"NOTE: That if this is specified as `dtk-ctk`, the PATH will be searched for `dtk-ctk` instead.",
Name: "dcu-cdi-hook-path",
Aliases: []string{"dcu-ctk-path"},
Usage: "Specify the path to use for the dcu-cdi-hook in the generated CDI specification. " +
"If not specified, the PATH will be searched for `dcu-cdi-hook`. " +
"NOTE: That if this is specified as `dcu-ctk`, the PATH will be searched for `dcu-ctk` instead.",
Destination: &opts.dtkCDIHookPath,
},
&cli.StringFlag{
......
......@@ -5,7 +5,7 @@
package list
import (
"dtk-container-toolkit/internal/logger"
"dcu-container-toolkit/internal/logger"
"errors"
"fmt"
......
......@@ -5,9 +5,9 @@
package root
import (
"dtk-container-toolkit/internal/logger"
"dtk-container-toolkit/pkg/c3000cdi/spec"
transformroot "dtk-container-toolkit/pkg/c3000cdi/transform/root"
"dcu-container-toolkit/internal/logger"
"dcu-container-toolkit/pkg/c3000cdi/spec"
transformroot "dcu-container-toolkit/pkg/c3000cdi/transform/root"
"fmt"
"io"
"os"
......
......@@ -5,8 +5,8 @@
package transform
import (
"dtk-container-toolkit/cmd/dtk-ctk/cdi/transform/root"
"dtk-container-toolkit/internal/logger"
"dcu-container-toolkit/cmd/dcu-ctk/cdi/transform/root"
"dcu-container-toolkit/internal/logger"
"github.com/urfave/cli/v2"
)
......
......@@ -5,10 +5,10 @@
package config
import (
createdefault "dtk-container-toolkit/cmd/dtk-ctk/config/create-default"
"dtk-container-toolkit/cmd/dtk-ctk/config/flags"
"dtk-container-toolkit/internal/config"
"dtk-container-toolkit/internal/logger"
createdefault "dcu-container-toolkit/cmd/dcu-ctk/config/create-default"
"dcu-container-toolkit/cmd/dcu-ctk/config/flags"
"dcu-container-toolkit/internal/config"
"dcu-container-toolkit/internal/logger"
"errors"
"fmt"
"reflect"
......
......@@ -5,9 +5,9 @@
package defaultsubcommand
import (
"dtk-container-toolkit/cmd/dtk-ctk/config/flags"
"dtk-container-toolkit/internal/config"
"dtk-container-toolkit/internal/logger"
"dcu-container-toolkit/cmd/dcu-ctk/config/flags"
"dcu-container-toolkit/internal/config"
"dcu-container-toolkit/internal/logger"
"fmt"
"github.com/urfave/cli/v2"
......
package dcuTracker
import (
"dtk-container-toolkit/cmd/dtk-ctk/dcu-tracker/disable"
"dtk-container-toolkit/cmd/dtk-ctk/dcu-tracker/enable"
"dtk-container-toolkit/cmd/dtk-ctk/dcu-tracker/initialize"
"dtk-container-toolkit/cmd/dtk-ctk/dcu-tracker/release"
"dtk-container-toolkit/cmd/dtk-ctk/dcu-tracker/reset"
"dtk-container-toolkit/cmd/dtk-ctk/dcu-tracker/status"
"dtk-container-toolkit/internal/dcu-tracker"
"dtk-container-toolkit/internal/logger"
"dcu-container-toolkit/cmd/dcu-ctk/dcu-tracker/disable"
"dcu-container-toolkit/cmd/dcu-ctk/dcu-tracker/enable"
"dcu-container-toolkit/cmd/dcu-ctk/dcu-tracker/initialize"
"dcu-container-toolkit/cmd/dcu-ctk/dcu-tracker/release"
"dcu-container-toolkit/cmd/dcu-ctk/dcu-tracker/reset"
"dcu-container-toolkit/cmd/dcu-ctk/dcu-tracker/status"
"dcu-container-toolkit/internal/dcu-tracker"
"dcu-container-toolkit/internal/logger"
"fmt"
"os/user"
......@@ -30,19 +30,19 @@ func (m command) build() *cli.Command {
dcuTrackerCmd := cli.Command{
Name: "dcu-tracker",
Usage: "DCU Tracker related commands",
UsageText: `dtk-ctk dcu-tracker [dcu-ids] [accessibility]
UsageText: `dcu-ctk dcu-tracker [dcu-ids] [accessibility]
Arguments:
dcu-ids Comma-separated list of DCU IDs (comma separated list, range operator, all)
accessibility Must be either 'exclusive' or 'shared'
Examples:
dtk-ctk dcu-tracker 0,1,2 exclusive
dtk-ctk dcu-tracker 0,1-2 shared
dtk-ctk dcu-tracker all shared
dcu-ctk dcu-tracker 0,1,2 exclusive
dcu-ctk dcu-tracker 0,1-2 shared
dcu-ctk dcu-tracker all shared
OR
dtk-ctk dcu-tracker [command] [options]`,
dcu-ctk dcu-tracker [command] [options]`,
Before: func(c *cli.Context) error { return m.validateGenOptions(c) },
Action: func(c *cli.Context) error { return m.performAction(c) },
}
......
package disable
import (
dcuTracker "dtk-container-toolkit/internal/dcu-tracker"
dcuTracker "dcu-container-toolkit/internal/dcu-tracker"
"fmt"
"os/user"
"dtk-container-toolkit/internal/logger"
"dcu-container-toolkit/internal/logger"
"github.com/urfave/cli/v2"
)
......@@ -26,7 +26,7 @@ func (m command) build() *cli.Command {
dcuTrackerDisableCmd := cli.Command{
Name: "disable",
Usage: "Disable the DCU Tracker",
UsageText: "dtk-ctk dcu-tracker disable [options]",
UsageText: "dcu-ctk dcu-tracker disable [options]",
Before: func(c *cli.Context) error { return validateGenOptions(c) },
Action: func(c *cli.Context) error { return performAction(c) },
}
......
package enable
import (
dcuTracker "dtk-container-toolkit/internal/dcu-tracker"
dcuTracker "dcu-container-toolkit/internal/dcu-tracker"
"fmt"
"os/user"
"dtk-container-toolkit/internal/logger"
"dcu-container-toolkit/internal/logger"
"github.com/urfave/cli/v2"
)
......@@ -26,7 +26,7 @@ func (m command) build() *cli.Command {
dcuTrackerEnableCmd := cli.Command{
Name: "enable",
Usage: "Enable the DCU Tracker",
UsageText: "dtk-ctk dcu-tracker enable [options]",
UsageText: "dcu-ctk dcu-tracker enable [options]",
Before: func(c *cli.Context) error { return validateGenOptions(c) },
Action: func(c *cli.Context) error { return performAction(c) },
}
......
package initialize
import (
dcuTracker "dtk-container-toolkit/internal/dcu-tracker"
dcuTracker "dcu-container-toolkit/internal/dcu-tracker"
"fmt"
"os/user"
"dtk-container-toolkit/internal/logger"
"dcu-container-toolkit/internal/logger"
"github.com/urfave/cli/v2"
)
......@@ -27,7 +27,7 @@ func (m command) build() *cli.Command {
Name: "init",
Hidden: true,
Usage: "Initialize the DCU Tracker",
UsageText: "dtk-ctk dcu-tracker init [options]",
UsageText: "dcu-ctk dcu-tracker init [options]",
Before: func(c *cli.Context) error { return validateGenOptions(c) },
Action: func(c *cli.Context) error { return performAction(c) },
}
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment