"ict_eval_bm25.py" did not exist on "dfb907fe557c5fe8f46c2501b4a87ed4e1bd9c46"
README.md 2.09 KB
Newer Older
zhanggzh's avatar
zhanggzh committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
# KerasCV Accelerators Testing

This `cloudbuild/` directory contains configurations for accelerators (GPU/TPU)
testing. Briefly, for each PR, it copies the PR's code to a base docker image
which contains KerasCV dependencies to make a new docker image, and deploys the
new image to Google Kubernetes Engine cluster, then run all tests in
`keras_cv/` via Google Cloud Build.

- `cloudbuild.yaml`: The cloud build configuration that specifies steps to run
  by cloud build.
- `Dockerfile`: The configuration to build the docker image for deployment.
- `requirements.txt`: Dependencies of KerasCV.
- `unit_test_jobs.jsonnet`: Jsonnet config that tells GKE cluster to run all
  unit tests in `keras_cv/`.

This test is powered by [ml-testing-accelerators](https://github.com/GoogleCloudPlatform/ml-testing-accelerators).


### Adding Test Dependencies
You must be authorized to run builds in the `keras-team-test` GCP project.
If you are not, please open a GitHub issue and ping a team member.
To authorize yourself with `keras-team-test`, run:

```bash
gcloud config set project keras-team-test
```

To add a dependency for GPU tests:
- Create a PR adding the dependency to `requirements.txt`
- Have a Keras team member update the Docker image for GPU tests by running the remaining steps
- Create a `Dockerfile` with the following contents:
```
FROM tensorflow/tensorflow:2.10.0-gpu
RUN \
    apt-get -y update && \
    apt-get -y install openjdk-8-jdk && \
    echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list && \
    curl https://bazel.build/bazel-release.pub.gpg | apt-key add
RUN apt-get -y update
RUN apt-get -y install bazel
RUN apt-get -y install git
RUN git clone https://github.com/{path_to_keras_cv_fork}.git
RUN cd keras-cv && git checkout {branch_name}
RUN pip install -r keras-cv/cloudbuild/requirements.txt
```
- Run the following command from the directory with your `Dockerfile`:
```
gcloud builds submit --region=us-west1 --tag us-west1-docker.pkg.dev/keras-team-test/keras-cv-test/keras-cv-image:deps --timeout=10m
```
- Merge the PR adding the dependency