Skip to content

GitLab

Unverified Commit 5dbfcdc4 authored by James Lamb's avatar James Lamb Committed by GitHub
Browse files

[ci] use GitHub built-in token, switch comment-triggered jobs to workflow... 


[ci] use GitHub built-in token, switch comment-triggered jobs to workflow dispatch  (fixes #7012) (#7035)
Co-authored-by: default avatargithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
parent 2c6b74af
Hide whitespace changes
Inline Side-by-side
Showing with 146 additions and 255 deletions
.ci/append-comment.sh
View file @ 5dbfcdc4
#!/bin/bash
#
# [description]
# Update comment appending a given body to the specified original comment.
# Post a comment to a pull request.
#
# [usage]
# append-comment.sh <COMMENT_ID> <BODY>
# append-comment.sh <PULL_REQUEST_ID> <BODY>
#
# COMMENT_ID: ID of comment that should be modified.
# PULL_REQUEST_ID: ID of PR to post the comment on.
#
# BODY: Text that will be appended to the original comment body.
# BODY: Text of the comment to be posted.
set -e -E -u -o pipefail
......@@ -18,20 +18,13 @@ if [ -z "$GITHUB_ACTIONS" ]; then
fi
if [ $# -ne 2 ]; then
echo "Usage: $0 <COMMENT_ID> <BODY>"
echo "Usage: $0 <PULL_REQUEST_ID> <BODY>"
exit 1
fi
comment_id=$1
pr_id=$1
body=$2
old_comment_body=$(
curl -sL \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $SECRETS_WORKFLOW" \
"${GITHUB_API_URL}/repos/microsoft/LightGBM/issues/comments/$comment_id" | \
jq '.body'
)
body=${body/failure/failure ❌}
body=${body/error/failure ❌}
body=${body/cancelled/failure ❌}
......@@ -39,12 +32,13 @@ body=${body/timed_out/failure ❌}
body=${body/success/success ✔️}
data=$(
jq -n \
--argjson body "${old_comment_body%?}\r\n\r\n$body\"" \
'{"body":$body}'
--argjson body "\"$body\"" \
'{"body": $body}'
)
curl -sL \
-X PATCH \
--fail \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $SECRETS_WORKFLOW" \
-H "Authorization: token ${GITHUB_TOKEN}" \
-d "$data" \
"${GITHUB_API_URL}/repos/microsoft/LightGBM/issues/comments/$comment_id"
"${GITHUB_API_URL}/repos/microsoft/LightGBM/issues/${pr_id}/comments"
.ci/check-workflow-status.sh 0 → 100755
View file @ 5dbfcdc4
#!/bin/bash
# [description]
#
# Look for the last run of a given GitHub Actions workflow on a given branch.
# If there's never been one (as might be the case with optional workflows like valgrind),
# exit with 0.
#
# Otherwise, check the status of that latest run.
# If it wasn't successful, exit with a non-0 exit code.
#
# [usage]
#
# check-workflow-status.sh <BRANCH> <WORKFLOW_FILE>
#
# BRANCH: name of a branch involved in a pull request.
#
# WORKFLOW_FILE: filename (e.g. 'r_valgrind.yml') defining the GitHub Actions workflow.
#
set -e -u -o pipefail
BRANCH="${1}"
WORKFLOW_FILE="${2}"
echo "Searching for latest run of '${WORKFLOW_FILE}' on branch '${BRANCH}'"
LATEST_RUN_ID=$(
gh run list \
--repo "microsoft/LightGBM" \
--branch "${BRANCH}" \
--workflow "${WORKFLOW_FILE}" \
--json 'createdAt,databaseId' \
--jq 'sort_by(.createdAt) | reverse | .[0] | .databaseId'
)
if [[ "${LATEST_RUN_ID}" == "" ]]; then
echo "No runs of '${WORKFLOW_FILE}' found on branch '${BRANCH}'"
exit 0
fi
echo "Checking status of workflow run '${LATEST_RUN_ID}'"
gh run view \
--repo "microsoft/LightGBM" \
--exit-status \
"${LATEST_RUN_ID}"
.ci/get-workflow-status.py deleted 100644 → 0
View file @ 2c6b74af
# coding: utf-8
"""Get the most recent status of workflow for the current PR.
[usage]
python get-workflow-status.py TRIGGER_PHRASE
TRIGGER_PHRASE: Code phrase that triggers workflow.
"""
import json
from os import environ
from sys import argv, exit
from time import sleep
from urllib import request
def get_runs(trigger_phrase):
"""Get all triggering workflow comments in the current PR.
Parameters
----------
trigger_phrase : str
Code phrase that triggers workflow.
Returns
-------
pr_runs : list
List of comment objects sorted by the time of creation in decreasing order.
"""
pr_runs = []
if environ.get("GITHUB_EVENT_NAME", "") == "pull_request":
pr_number = int(environ.get("GITHUB_REF").split("/")[-2])
page = 1
while True:
req = request.Request(
url="{}/repos/microsoft/LightGBM/issues/{}/comments?page={}&per_page=100".format(
environ.get("GITHUB_API_URL"), pr_number, page
),
headers={"Accept": "application/vnd.github.v3+json"},
)
url = request.urlopen(req)
data = json.loads(url.read().decode("utf-8"))
url.close()
if not data:
break
runs_on_page = [
i
for i in data
if i["author_association"].lower() in {"owner", "member", "collaborator"}
and i["body"].startswith("/gha run {}".format(trigger_phrase))
]
pr_runs.extend(runs_on_page)
page += 1
return pr_runs[::-1]
def get_status(runs):
"""Get the most recent status of workflow for the current PR.
Parameters
----------
runs : list
List of comment objects sorted by the time of creation in decreasing order.
Returns
-------
status : str
The most recent status of workflow.
Can be 'success', 'failure' or 'in-progress'.
"""
status = "success"
for run in runs:
body = run["body"]
if "Status: " in body:
if "Status: skipped" in body:
continue
if "Status: failure" in body:
status = "failure"
break
if "Status: success" in body:
status = "success"
break
else:
status = "in-progress"
break
return status
if __name__ == "__main__":
trigger_phrase = argv[1]
while True:
status = get_status(get_runs(trigger_phrase))
if status != "in-progress":
break
sleep(60)
if status == "failure":
exit(1)
.ci/rerun-workflow.sh
View file @ 5dbfcdc4
......@@ -31,7 +31,7 @@ pr_branch=$3
runs=$(
curl -sL \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $SECRETS_WORKFLOW" \
-H "Authorization: token ${GITHUB_TOKEN}" \
"${GITHUB_API_URL}/repos/microsoft/LightGBM/actions/workflows/${workflow_id}/runs?event=pull_request&branch=${pr_branch}" | \
jq '.workflow_runs'
)
......@@ -42,6 +42,6 @@ if [[ $(echo "${runs}" | jq 'length') -gt 0 ]]; then
curl -sL \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $SECRETS_WORKFLOW" \
-H "Authorization: token ${GITHUB_TOKEN}" \
"${GITHUB_API_URL}/repos/microsoft/LightGBM/actions/runs/$(echo "${runs}" | jq '.[0].id')/rerun"
fi
.ci/set-commit-status.sh
View file @ 5dbfcdc4
......@@ -46,8 +46,9 @@ data=$(
)
curl -sL \
--fail \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $SECRETS_WORKFLOW" \
-H "Authorization: token ${GITHUB_TOKEN}" \
-d "$data" \
"${GITHUB_API_URL}/repos/microsoft/LightGBM/statuses/$sha"
.ci/trigger-dispatch-run.sh deleted 100755 → 0
View file @ 2c6b74af
#!/bin/bash
#
# [description]
# Trigger manual workflow run by a dispatch event.
#
# [usage]
# trigger-dispatch-run.sh <PR_URL> <COMMENT_ID> <DISPATCH_NAME>
#
# PR_URL: URL of pull request from which dispatch is triggering.
#
# COMMENT_ID: ID of comment that is triggering a dispatch.
#
# DISPATCH_NAME: Name of a dispatch to be triggered.
set -e -E -u -o pipefail
if [ -z "$GITHUB_ACTIONS" ]; then
echo "Must be run inside GitHub Actions CI"
exit 1
fi
if [ $# -ne 3 ]; then
echo "Usage: $0 <PR_URL> <COMMENT_ID> <DISPATCH_NAME>"
exit 1
fi
pr_url=$1
comment_id=$2
dispatch_name=$3
pr=$(
curl -sL \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $SECRETS_WORKFLOW" \
"$pr_url"
)
data=$(
jq -n \
--arg event_type "$dispatch_name" \
--arg pr_number "$(echo "$pr" | jq '.number')" \
--arg pr_sha "$(echo "$pr" | jq '.head.sha')" \
--arg pr_branch "$(echo "$pr" | jq '.head.ref')" \
--arg comment_number "$comment_id" \
'{"event_type":$event_type,"client_payload":{"pr_number":$pr_number,"pr_sha":$pr_sha,"pr_branch":$pr_branch,"comment_number":$comment_number}}'
)
curl -sL \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $SECRETS_WORKFLOW" \
-d "$data" \
"${GITHUB_API_URL}/repos/microsoft/LightGBM/dispatches"
.github/workflows/optional_checks.yml
View file @ 5dbfcdc4
......@@ -7,8 +7,10 @@ on:
jobs:
all-optional-checks-successful:
timeout-minutes: 120
timeout-minutes: 30
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ github.token }}
steps:
- name: Checkout repository
uses: actions/checkout@v5
......@@ -16,19 +18,12 @@ jobs:
fetch-depth: 5
persist-credentials: false
submodules: false
- name: Check that all tests succeeded
- name: Check valgrind workflow
shell: bash
run: |
workflows=(
"R valgrind tests;r-valgrind"
)
for i in "${workflows[@]}"; do
workflow_name=${i%;*}
comment="The last reported status from workflow \"$workflow_name\" is failure."
comment="${comment} Commit fixes and rerun the workflow."
trigger_phrase=${i#*;}
python \
"$GITHUB_WORKSPACE/.ci/get-workflow-status.py" \
"$trigger_phrase" \
|| { echo "${comment}"; exit 1; }
done
# ref: https://docs.github.com/en/actions/reference/workflows-and-actions/contexts#github-context
PR_BRANCH="${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}"
echo "checking status for branch '${PR_BRANCH}'"
./.ci/check-workflow-status.sh \
"${PR_BRANCH}" \
'r_valgrind.yml'
.github/workflows/r_configure.yml
View file @ 5dbfcdc4
name: R generate configure
on:
repository_dispatch:
types: [gha_run_r_configure]
workflow_dispatch:
inputs:
pr-branch:
type: string
description: |
Branch in microsoft/LightGBM to update.
permissions:
actions: none
checks: none
contents: write
deployments: none
discussions: none
id-token: write
issues: none
packages: none
pages: none
pull-requests: read
repository-projects: none
security-events: none
statuses: none
jobs:
r-configure:
......@@ -24,18 +43,19 @@ jobs:
uses: actions/checkout@v5
with:
fetch-depth: 5
submodules: true
submodules: false
repository: microsoft/LightGBM
ref: "refs/heads/${{ fromJSON(github.event.client_payload.pr_branch) }}"
token: ${{ secrets.WORKFLOW }}
ref: "refs/heads/${{ inputs.pr-branch }}"
token: ${{ github.token }}
persist-credentials: true
- name: Update configure
shell: bash
run: ./R-package/recreate-configure.sh || exit 1
- name: Push changes
run: |
git config --global user.name "GitHub Actions Bot"
git config --global user.email "githubactionsbot@users.noreply.github.com"
# source for this user and email: https://github.com/orgs/community/discussions/160496
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git add "./R-package/configure"
git commit --allow-empty -m "Auto-update configure"
git push
.github/workflows/r_valgrind.yml
View file @ 5dbfcdc4
name: R valgrind tests
on:
repository_dispatch:
types: [gha_run_r_valgrind]
workflow_dispatch:
inputs:
pr-branch:
type: string
description: |
Branch the PR was submitted from.
Branches from forks should be prefixed with the user/org they originate from,
like '{user}:{branch}'.
pr-number:
type: string
description: Pull request ID, found in the PR URL.
permissions:
actions: none
checks: write
contents: read
deployments: none
discussions: none
id-token: write
issues: none
packages: none
pages: none
pull-requests: write
repository-projects: none
security-events: none
statuses: write
jobs:
test-r-valgrind:
......@@ -11,7 +35,7 @@ jobs:
runs-on: ubuntu-latest
container: wch1/r-debug
env:
SECRETS_WORKFLOW: ${{ secrets.WORKFLOW }}
GITHUB_TOKEN: ${{ github.token }}
steps:
- name: Install essential software before checkout
shell: bash
......@@ -30,20 +54,7 @@ jobs:
submodules: true
persist-credentials: false
repository: microsoft/LightGBM
ref: "refs/pull/${{ github.event.client_payload.pr_number }}/merge"
- name: Send init status
if: ${{ always() }}
shell: bash
run: |
$GITHUB_WORKSPACE/.ci/set-commit-status.sh \
"${{ github.workflow }}" \
"pending" \
"${{ github.event.client_payload.pr_sha }}"
comment="Workflow **${{ github.workflow }}** has been triggered! 🚀\r\n"
comment="${comment}${GITHUB_SERVER_URL}/microsoft/LightGBM/actions/runs/${GITHUB_RUN_ID}"
$GITHUB_WORKSPACE/.ci/append-comment.sh \
"${{ github.event.client_payload.comment_number }}" \
"${comment}"
ref: "refs/pull/${{ inputs.pr-number }}/merge"
- name: Run tests with valgrind
shell: bash
run: ./.ci/test-r-package-valgrind.sh
......@@ -53,15 +64,18 @@ jobs:
$GITHUB_WORKSPACE/.ci/set-commit-status.sh \
"${{ github.workflow }}" \
"${{ job.status }}" \
"${{ github.event.client_payload.pr_sha }}"
"${{ github.sha }}"
comment="Workflow **${{ github.workflow }}** has been triggered! 🚀\r\n"
comment="${comment}\r\n${GITHUB_SERVER_URL}/microsoft/LightGBM/actions/runs/${GITHUB_RUN_ID} \r\n"
comment="${comment}\r\nStatus: ${{ job.status }}"
$GITHUB_WORKSPACE/.ci/append-comment.sh \
"${{ github.event.client_payload.comment_number }}" \
"Status: ${{ job.status }}."
"${{ inputs.pr-number }}" \
"${comment}"
- name: Rerun workflow-indicator
if: ${{ always() }}
run: |
bash $GITHUB_WORKSPACE/.ci/rerun-workflow.sh \
"optional_checks.yml" \
"${{ github.event.client_payload.pr_number }}" \
"${{ github.event.client_payload.pr_branch }}" \
"${{ inputs.pr-number }}" \
"${{ inputs.pr-branch }}" \
|| true
.github/workflows/triggering_comments.yml deleted 100644 → 0
View file @ 2c6b74af
name: Triggering comments
on:
issue_comment:
types: [created]
jobs:
triggering-tests:
if: |
github.event.issue.pull_request &&
contains('OWNER,MEMBER,COLLABORATOR', github.event.comment.author_association) &&
startsWith(github.event.comment.body, '/gha run')
runs-on: ubuntu-latest
env:
SECRETS_WORKFLOW: ${{ secrets.WORKFLOW }}
steps:
- name: Checkout repository
uses: actions/checkout@v5
with:
fetch-depth: 5
persist-credentials: false
submodules: false
- name: Trigger R valgrind tests
if: github.event.comment.body == '/gha run r-valgrind'
run: |
$GITHUB_WORKSPACE/.ci/trigger-dispatch-run.sh \
"${{ github.event.issue.pull_request.url }}" \
"${{ github.event.comment.id }}" \
"gha_run_r_valgrind"
- name: Trigger update R configure
if: github.event.comment.body == '/gha run r-configure'
run: |
$GITHUB_WORKSPACE/.ci/trigger-dispatch-run.sh \
"${{ github.event.issue.pull_request.url }}" \
"${{ github.event.comment.id }}" \
"gha_run_r_configure"
R-package/README.md
View file @ 5dbfcdc4
......@@ -378,9 +378,12 @@ At build time, `configure` will be run and used to create a file `Makevars`, usi
3. Edit `src/Makevars.in`.
Alternatively, GitHub Actions can re-generate this file for you. On a pull request (only on internal one, does not work for ones from forks), create a comment with this phrase:
Alternatively, GitHub Actions can re-generate this file for you.
> /gha run r-configure
1. navigate to https://github.com/microsoft/LightGBM/actions/workflows/r_configure.yml
2. click "Run workflow" (drop-down)
3. enter the branch from the pull request for the `pr-branch` input
4. click "Run workflow" (button)
**Configuring for Windows**
......@@ -483,9 +486,13 @@ RDvalgrind \
| cat
```
These tests can also be triggered on any pull request by leaving a comment in a pull request:
These tests can also be triggered on a pull request branch, using GitHub Actions.
> /gha run r-valgrind
1. navigate to https://github.com/microsoft/LightGBM/actions/workflows/r_valgrind.yml
2. click "Run workflow" (drop-down)
3. enter the branch from the pull request for the `pr-branch` input
4. enter the pull request ID for the `pr-number` input
5. click "Run workflow" (button)
Known Issues
------------
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment