-
Notifications
You must be signed in to change notification settings - Fork 37
Pin and verify SHAs for CI image dependencies #167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: windows-2019
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| name: Bump Binaries | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '0 4 * * 1' # Run every Monday at 4:00 AM UTC | ||
| workflow_dispatch: # Allow manual triggering | ||
|
|
||
| jobs: | ||
| bump-binaries: | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup jq | ||
| run: sudo apt-get install -y jq | ||
|
|
||
| - name: Run bump script | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| ./ci/scripts/bump-binaries.sh | ||
|
|
||
| - name: Test Docker build | ||
| run: | | ||
| # Mock the azstemcell binary for the docker build context | ||
| touch ci/docker/azstemcell | ||
|
|
||
| # Extract args from dependencies.json | ||
| DEPS_FILE="ci/docker/dependencies.json" | ||
| BOSH_CLI_URL=$(jq -r '.bosh_cli.url' < $DEPS_FILE) | ||
| BOSH_CLI_SHA=$(jq -r '.bosh_cli.sha256' < $DEPS_FILE) | ||
| META4_CLI_URL=$(jq -r '.meta4_cli.url' < $DEPS_FILE) | ||
| META4_CLI_SHA=$(jq -r '.meta4_cli.sha256' < $DEPS_FILE) | ||
| YQ_CLI_URL=$(jq -r '.yq_cli.url' < $DEPS_FILE) | ||
| YQ_CLI_SHA=$(jq -r '.yq_cli.sha256' < $DEPS_FILE) | ||
| RUBY_INSTALL_URL=$(jq -r '.ruby_install.url' < $DEPS_FILE) | ||
| RUBY_INSTALL_SHA=$(jq -r '.ruby_install.sha256' < $DEPS_FILE) | ||
| GOLANGCI_LINT_URL=$(jq -r '.golangci_lint.url' < $DEPS_FILE) | ||
| GOLANGCI_LINT_SHA=$(jq -r '.golangci_lint.sha256' < $DEPS_FILE) | ||
| GOVC_URL=$(jq -r '.govc.url' < $DEPS_FILE) | ||
| GOVC_SHA=$(jq -r '.govc.sha256' < $DEPS_FILE) | ||
|
|
||
| RUBY_VERSION=$(cat .ruby-version) | ||
| GEM_HOME="/usr/local/bundle" | ||
|
|
||
| docker build \ | ||
| --build-arg BOSH_CLI_URL="$BOSH_CLI_URL" \ | ||
| --build-arg BOSH_CLI_SHA256="$BOSH_CLI_SHA" \ | ||
| --build-arg META4_CLI_URL="$META4_CLI_URL" \ | ||
| --build-arg META4_CLI_SHA256="$META4_CLI_SHA" \ | ||
| --build-arg YQ_CLI_URL="$YQ_CLI_URL" \ | ||
| --build-arg YQ_CLI_SHA256="$YQ_CLI_SHA" \ | ||
| --build-arg RUBY_INSTALL_URL="$RUBY_INSTALL_URL" \ | ||
| --build-arg RUBY_INSTALL_SHA256="$RUBY_INSTALL_SHA" \ | ||
| --build-arg GOLANGCI_LINT_INSTALL_URL="$GOLANGCI_LINT_URL" \ | ||
| --build-arg GOLANGCI_LINT_INSTALL_SHA256="$GOLANGCI_LINT_SHA" \ | ||
| --build-arg GOVC_INSTALL_URL="$GOVC_URL" \ | ||
| --build-arg GOVC_INSTALL_SHA256="$GOVC_SHA" \ | ||
| --build-arg RUBY_VERSION="$RUBY_VERSION" \ | ||
| --build-arg GEM_HOME="$GEM_HOME" \ | ||
| -f ci/docker/Dockerfile ci/docker/ | ||
|
|
||
| - name: Create Pull Request | ||
| uses: peter-evans/create-pull-request@c5a7806660adbe173f04e3e038b0ccdcd758773c # v6.1.0 | ||
| with: | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
| commit-message: "Update binary dependencies in dependencies.json" | ||
| title: "Update binary dependencies" | ||
| body: | | ||
| This PR automatically updates the binary dependencies in `ci/docker/dependencies.json`. | ||
|
|
||
| This ensures that the CI image is always built with the latest versions of these tools while maintaining security via SHA-256 verification. | ||
| branch: "bump-binaries" | ||
| base: "windows-2019" | ||
| delete-branch: true | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| { | ||
| "bosh_cli": { | ||
| "url": "https://github.com/cloudfoundry/bosh-cli/releases/download/v7.10.2/bosh-cli-7.10.2-linux-amd64", | ||
| "sha256": "d9a52693994bdefc2fc73f1fe16042ffed48c6931308c6a9ae3520413d73065c" | ||
| }, | ||
| "meta4_cli": { | ||
| "url": "https://github.com/dpb587/metalink/releases/download/v0.5.0/meta4-0.5.0-linux-amd64", | ||
| "sha256": "9f3ff22e1ac3a8b4a667a9505dce2a224e099475ab69a02b23813ad073e27e01" | ||
| }, | ||
| "yq_cli": { | ||
| "url": "https://github.com/mikefarah/yq/releases/download/v4.53.2/yq_linux_amd64", | ||
| "sha256": "d56bf5c6819e8e696340c312bd70f849dc1678a7cda9c2ad63eebd906371d56b" | ||
| }, | ||
| "ruby_install": { | ||
| "url": "https://github.com/postmodern/ruby-install/releases/download/v0.10.2/ruby-install-0.10.2.tar.gz", | ||
| "sha256": "65836158b8026992b2e96ed344f3d888112b2b105d0166ecb08ba3b4a0d91bf6" | ||
| }, | ||
| "golangci_lint": { | ||
| "url": "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-linux-amd64.tar.gz", | ||
| "sha256": "200c5b7503f67b59a6743ccf32133026c174e272b930ee79aa2aa6f37aca7ef1" | ||
| }, | ||
| "govc": { | ||
| "url": "https://github.com/vmware/govmomi/releases/download/v0.53.0/govc_Linux_x86_64.tar.gz", | ||
| "sha256": "67cd529a9a2ec4c68fd1ae99cd4dcbee086591b029e23934a50564480e86e739" | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,90 @@ | ||||||||||||||||||||||
| #!/usr/bin/env bash | ||||||||||||||||||||||
| set -euo pipefail | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| # Find the absolute path to the dependencies.json file | ||||||||||||||||||||||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||||||||||||||||||||||
| DEPS_FILE="${SCRIPT_DIR}/../docker/dependencies.json" | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| TMP_FILE=$(mktemp) | ||||||||||||||||||||||
| trap 'rm -f "${TMP_FILE}"' EXIT | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| if ! command -v jq &> /dev/null; then | ||||||||||||||||||||||
| echo "Error: jq is required but not installed." >&2 | ||||||||||||||||||||||
| exit 1 | ||||||||||||||||||||||
| fi | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| SAFETY_DAYS=7 | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| get_latest_release_json() { | ||||||||||||||||||||||
| local repo=$1 | ||||||||||||||||||||||
| if command -v gh &> /dev/null; then | ||||||||||||||||||||||
| gh api "repos/${repo}/releases/latest" | ||||||||||||||||||||||
| else | ||||||||||||||||||||||
| curl -s -H "Authorization: token ${GITHUB_TOKEN:-}" "https://api.github.com/repos/${repo}/releases/latest" | ||||||||||||||||||||||
| fi | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| get_sha256() { | ||||||||||||||||||||||
| local url=$1 | ||||||||||||||||||||||
| curl -fsSL "${url}" | sha256sum | awk '{print $1}' | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| update_dep() { | ||||||||||||||||||||||
| local key=$1 | ||||||||||||||||||||||
| local repo=$2 | ||||||||||||||||||||||
| local pattern=$3 | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| echo "Checking ${repo}..." | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| local release_json | ||||||||||||||||||||||
| release_json=$(get_latest_release_json "${repo}") | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| # 1. Check the Safety Buffer using jq | ||||||||||||||||||||||
| local age_days | ||||||||||||||||||||||
| age_days=$(echo "${release_json}" | jq -r 'if .published_at then ((now - (.published_at | fromdateiso8601)) / 86400) | floor else -1 end') | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| if [[ "${age_days}" != "-1" ]] && [[ "${age_days}" -lt "${SAFETY_DAYS}" ]]; then | ||||||||||||||||||||||
| echo " Skipping: Release is only ${age_days} days old (requires ${SAFETY_DAYS} days of safety)." | ||||||||||||||||||||||
| return 0 | ||||||||||||||||||||||
| fi | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| # 2. Extract the URL | ||||||||||||||||||||||
| local url | ||||||||||||||||||||||
| url=$(echo "${release_json}" | jq -r ".assets[] | select(.name | test(\"${pattern}\")) | .browser_download_url" | head -n 1) | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
|
Comment on lines
+51
to
+54
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Avoid If multiple assets match, Proposed fix- url=$(echo "${release_json}" | jq -r ".assets[] | select(.name | test(\"${pattern}\")) | .browser_download_url" | head -n 1)
+ url=$(echo "${release_json}" | jq -r --arg pattern "${pattern}" '
+ first(.assets[] | select(.name | test($pattern)) | .browser_download_url) // empty
+ ')📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||
| if [[ -z "${url}" || "${url}" == "null" ]]; then | ||||||||||||||||||||||
| echo "Failed to find asset for ${repo} matching ${pattern}" >&2 | ||||||||||||||||||||||
| return 1 | ||||||||||||||||||||||
| fi | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| # 3. Compare with current version | ||||||||||||||||||||||
| local current_url | ||||||||||||||||||||||
| current_url=$(jq -r ".${key}.url" < "${DEPS_FILE}") | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| if [[ "${url}" == "${current_url}" ]]; then | ||||||||||||||||||||||
| echo " Already up to date." | ||||||||||||||||||||||
| return 0 | ||||||||||||||||||||||
| fi | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| # 4. Apply the update | ||||||||||||||||||||||
| echo " Found new version: ${url}" | ||||||||||||||||||||||
| echo " Calculating SHA256..." | ||||||||||||||||||||||
| local sha | ||||||||||||||||||||||
| sha=$(get_sha256 "${url}") | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| echo " Updating ${DEPS_FILE}..." | ||||||||||||||||||||||
| jq ".${key}.url = \"${url}\" | .${key}.sha256 = \"${sha}\"" "${DEPS_FILE}" > "${TMP_FILE}" | ||||||||||||||||||||||
| mv "${TMP_FILE}" "${DEPS_FILE}" | ||||||||||||||||||||||
|
Comment on lines
+75
to
+77
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick | 🔵 Trivial Pass updated values to Directly interpolating Proposed fix- jq ".${key}.url = \"${url}\" | .${key}.sha256 = \"${sha}\"" "${DEPS_FILE}" > "${TMP_FILE}"
+ jq --arg key "${key}" --arg url "${url}" --arg sha "${sha}" '
+ .[$key].url = $url | .[$key].sha256 = $sha
+ ' "${DEPS_FILE}" > "${TMP_FILE}"🤖 Prompt for AI Agents |
||||||||||||||||||||||
| echo " Done." | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| echo "Bumping binaries in ${DEPS_FILE}..." | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| update_dep "bosh_cli" "cloudfoundry/bosh-cli" "linux-amd64" | ||||||||||||||||||||||
| update_dep "meta4_cli" "dpb587/metalink" "meta4-[0-9]+.[0-9]+.[0-9]+-linux-amd64" | ||||||||||||||||||||||
| update_dep "yq_cli" "mikefarah/yq" "linux_amd64$" | ||||||||||||||||||||||
| update_dep "ruby_install" "postmodern/ruby-install" "tar.gz$" | ||||||||||||||||||||||
| update_dep "golangci_lint" "golangci/golangci-lint" "golangci-lint-[0-9]+.[0-9]+.[0-9]+-linux-amd64.tar.gz" | ||||||||||||||||||||||
| update_dep "govc" "vmware/govmomi" "govc_Linux_x86_64.tar.gz" | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| echo "All binaries bumped successfully!" | ||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: cloudfoundry/bosh-windows-stemcell-builder
Length of output: 1347
🏁 Script executed:
Repository: cloudfoundry/bosh-windows-stemcell-builder
Length of output: 3314
Remove the mock
azstemcellfile before creating the PR.Line 32 creates
ci/docker/azstemcellfor the Docker build, but it remains as an untracked, unignored file when the PR creation step executes. Without cleanup or path restriction, it will be committed to the pull request alongside the intendeddependencies.jsonchanges.Add
rm -f ci/docker/azstemcellafter the Docker build step (after line 67) and addadd-paths: ci/docker/dependencies.jsonto the PR action to ensure only the intended file is committed.Proposed fix
docker build \ --build-arg BOSH_CLI_URL="$BOSH_CLI_URL" \ --build-arg BOSH_CLI_SHA256="$BOSH_CLI_SHA" \ --build-arg META4_CLI_URL="$META4_CLI_URL" \ --build-arg META4_CLI_SHA256="$META4_CLI_SHA" \ --build-arg YQ_CLI_URL="$YQ_CLI_URL" \ --build-arg YQ_CLI_SHA256="$YQ_CLI_SHA" \ --build-arg RUBY_INSTALL_URL="$RUBY_INSTALL_URL" \ --build-arg RUBY_INSTALL_SHA256="$RUBY_INSTALL_SHA" \ --build-arg GOLANGCI_LINT_INSTALL_URL="$GOLANGCI_LINT_URL" \ --build-arg GOLANGCI_LINT_INSTALL_SHA256="$GOLANGCI_LINT_SHA" \ --build-arg GOVC_INSTALL_URL="$GOVC_URL" \ --build-arg GOVC_INSTALL_SHA256="$GOVC_SHA" \ --build-arg RUBY_VERSION="$RUBY_VERSION" \ --build-arg GEM_HOME="$GEM_HOME" \ -f ci/docker/Dockerfile ci/docker/ + + rm -f ci/docker/azstemcell - name: Create Pull Request uses: peter-evans/create-pull-request@c5a7806660adbe173f04e3e038b0ccdcd758773c # v6.1.0 with: token: ${{ secrets.GITHUB_TOKEN }} + add-paths: ci/docker/dependencies.json commit-message: "Update binary dependencies in dependencies.json"🤖 Prompt for AI Agents