Repository navigation
test(conformance): run conformance suites from test plans in tmachine environments #4629
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| name: E2E Suites | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| source-sha: | ||
| description: Source revision of the prepared candidate artifacts | ||
| required: true | ||
| type: string | ||
| integration-inputs-artifact-id: | ||
| description: Immutable ID of the prepared integration inputs artifact | ||
| required: true | ||
| type: string | ||
| matrix: | ||
| description: >- | ||
| JSON array of {"plan", "runner"} entries, each with an optional | ||
| "environment". A plan is a file stem in tests/plans/ that selects | ||
| the suites and environments to run; "environment" narrows an entry | ||
| to one of them, so environments run as parallel jobs. | ||
| required: true | ||
| type: string | ||
|
|
||
| permissions: | ||
| actions: read | ||
| contents: read | ||
| packages: read | ||
|
|
||
| # github.workflow is the caller's name here, so scope the group to this job | ||
| # to avoid colliding with (and cancelling) the calling workflow. | ||
| concurrency: | ||
| group: ${{ github.workflow }}-e2e-suites-${{ github.ref }} | ||
|
Member
There was a problem hiding this comment. The old runner keyed concurrency on |
||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| e2e-suites: | ||
| name: E2E suites (${{ matrix.plan }}, ${{ matrix.environment || matrix.runner }}) | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: ${{ fromJSON(inputs.matrix) }} | ||
| runs-on: ${{ matrix.runner }} | ||
| timeout-minutes: 90 | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| ref: ${{ inputs['source-sha'] }} | ||
| persist-credentials: false | ||
|
|
||
| - name: Enable KVM access | ||
| run: | | ||
| set -euo pipefail | ||
| if [[ ! -c /dev/kvm ]]; then | ||
| echo "::error::The runner did not expose /dev/kvm" | ||
| exit 1 | ||
| fi | ||
| sudo chmod 0666 /dev/kvm | ||
| exec 3<>/dev/kvm | ||
| exec 3>&- | ||
|
|
||
| - uses: ./.github/actions/setup-nix | ||
|
|
||
| - name: Cache tmachine disks | ||
| uses: actions/cache@caa296126883cff596d87d8935842f9db880ef25 # v5.1.0 | ||
| with: | ||
| path: ~/.cache/tmachine | ||
| key: tmachine-${{ runner.os }}-${{ runner.arch }} | ||
|
Member
There was a problem hiding this comment.
|
||
|
|
||
| - name: Download integration inputs | ||
| uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 | ||
| with: | ||
| artifact-ids: ${{ inputs['integration-inputs-artifact-id'] }} | ||
| path: artifacts | ||
|
|
||
| - name: Run suites | ||
| env: | ||
| PLAN: ${{ matrix.plan }} | ||
| ENVIRONMENT: ${{ matrix.environment || '' }} | ||
| run: | | ||
| # Retry dependency preparation, then run the plan once. The runner | ||
| # builds the suites for the guest, so it needs no prepared archives. | ||
| nix build --no-link .#test-runner || nix build --no-link .#test-runner | ||
| args=(test --plan "tests/plans/${PLAN}.yaml" --results artifacts/test-results) | ||
| if [[ -n "${ENVIRONMENT}" ]]; then | ||
| args+=(--env "${ENVIRONMENT}") | ||
| fi | ||
| nix run .#test-runner -- "${args[@]}" | ||
|
|
||
| - name: Upload results | ||
| if: always() | ||
| uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 | ||
| with: | ||
| name: e2e-results-${{ matrix.plan }}-${{ matrix.environment || matrix.runner }} | ||
| path: artifacts/test-results | ||
| if-no-files-found: ignore | ||
| retention-days: 7 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This list is copied by hand from
cpu-tests.yaml(5 of its 10 environments) and repeated in release-dev and release-qualification, with no check between them. The arm64 environments are planned but never run, and nothing reports it, which is the failure mode that hid Kubernetes coverage. I'd generate this from the plan, or check it against the plan, before cutover.