Skip to content

Commit cc693d2

Browse files
authored
Merge pull request #44461 from github/repo-sync
Repo sync
2 parents 61d3752 + 6cef15e commit cc693d2

109 files changed

Lines changed: 1253 additions & 60661 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

content/actions/tutorials/build-and-test-code/python.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -297,6 +297,13 @@ steps:
297297
pytest tests.py --doctest-modules --junitxml=junit/test-results.xml --cov=com --cov-report=xml --cov-report=html
298298
```
299299

300+
{% ifversion code-quality %}
301+
302+
> [!TIP]
303+
> This example already produces a Cobertura XML coverage report (`--cov-report=xml`). To display coverage results directly on pull requests, upload the report using the `actions/upload-code-coverage` action. See [AUTOTITLE](/code-security/how-tos/maintain-quality-code/set-up-code-coverage).
304+
305+
{% endif %}
306+
300307
### Using Ruff to lint and/or format code
301308

302309
The following example installs or upgrades `ruff` and uses it to lint all files. For more information, see [Ruff](https://docs.astral.sh/ruff).

content/actions/tutorials/store-and-share-data.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,13 @@ jobs:
8383
path: output/test/code-coverage.html
8484
```
8585
86+
{% ifversion code-quality %}
87+
88+
> [!TIP]
89+
> To display code coverage results directly on pull requests instead of downloading artifacts, you can upload a Cobertura XML coverage report to {% data variables.product.prodname_code_quality_short %}. See [AUTOTITLE](/code-security/how-tos/maintain-quality-code/set-up-code-coverage).
90+
91+
{% endif %}
92+
8693
## Configuring a custom artifact retention period
8794
8895
You can define a custom retention period for individual artifacts created by a workflow. When using a workflow to create a new artifact, you can use `retention-days` with the `upload-artifact` action. This example demonstrates how to set a custom retention period of 5 days for the artifact named `my-artifact`:

content/code-security/concepts/about-code-quality.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ With {% data variables.product.prodname_code_quality_short %}, you can:
3030
* Use **repository dashboards** to track reliability and maintainability scores, identify areas needing attention, and prioritize remediation.
3131
* Monitor **organization dashboards** to understand the code health of your repositories at a glance and determine which repositories to investigate further.
3232
* Set up **rulesets** for pull requests to enforce code quality standards and block changes that do not meet your criteria.
33+
* Upload **code coverage** reports to see test coverage metrics directly on pull requests, helping reviewers identify untested code.
3334
* Easily assign remediation work to **{% data variables.copilot.copilot_cloud_agent %}**, if you have a {% data variables.product.prodname_copilot_short %} license.
3435

3536
## Availability and usage costs
@@ -63,6 +64,8 @@ In addition, you'll see an AI-powered analysis of all recent pushes to the defau
6364

6465
When {% data variables.product.prodname_codeql %} finds rule-based problems on pull requests, you'll see comments from the `{% data variables.code-quality.pr_commenter %}`. Where possible, each comment will include a {% data variables.copilot.copilot_autofix_short %} suggestion on how to fix the problem. See [AUTOTITLE](/code-security/code-quality/tutorials/fix-findings-in-prs).
6566

67+
If you have set up code coverage, the `{% data variables.code-quality.pr_commenter %}` also posts a coverage summary showing the aggregate coverage percentage for the PR branch compared to the default branch. See [AUTOTITLE](/code-security/how-tos/maintain-quality-code/interpret-results#viewing-code-coverage-on-pull-requests).
68+
6669
### Default branch results
6770

6871
{% data variables.product.prodname_code_quality_short %} findings on the default branch are reported on "{% data variables.code-quality.code_quality_ui %}" pages on the **{% data variables.product.prodname_security_and_quality_tab %}** tab for the repository:

content/code-security/how-tos/maintain-quality-code/enable-code-quality.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,5 @@ category:
4242
## Next steps
4343

4444
* **For your repository:** Explore your code quality findings and merge your first fix. See [AUTOTITLE](/code-security/tutorials/improve-code-quality/quickstart).
45+
* **Add code coverage:** Upload test coverage reports to see coverage results directly on pull requests. See [AUTOTITLE](/code-security/how-tos/maintain-quality-code/set-up-code-coverage).
4546
* **For your organization:** Understand the code health of your repositories at a glance. See [AUTOTITLE](/code-security/how-tos/view-and-interpret-data/analyze-organization-data/explore-code-quality).

content/code-security/how-tos/maintain-quality-code/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ versions:
88
contentType: how-tos
99
children:
1010
- /enable-code-quality
11+
- /set-up-code-coverage
1112
- /interpret-results
1213
- /set-pr-thresholds
1314
- /unblock-your-pr

content/code-security/how-tos/maintain-quality-code/interpret-results.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,18 @@ Code quality results should always be interpreted in the context of your reposit
5454

5555
To learn more about the metrics and how the ratings are calculated, see [AUTOTITLE](/code-security/code-quality/reference/metrics-and-ratings).
5656

57+
## Viewing code coverage on pull requests
58+
59+
After code coverage is configured for your repository, the `{% data variables.code-quality.pr_commenter %}` posts a coverage summary comment on each pull request. The comment includes:
60+
61+
* **Branch-vs-default-branch comparison:** The aggregate coverage percentage for the pull request branch and the default branch (for example, "65% on pull request branch, 44% on default branch").
62+
* **Most impacted files:** The 10 files with the largest coverage changes between the default branch and the pull request branch. This list may include files not directly modified in the pull request. New or modified files are ranked above deleted files. Within each group, files are sorted by the absolute change in line coverage weighted by the number of lines in the file, with coverage magnitude and then filename as tiebreakers.
63+
* **Per-file breakdown:** An expandable section listing each file with its coverage percentage and delta value. A positive delta means the file gained coverage on this branch. A negative delta indicates coverage decreased, which may signal untested code paths introduced by the change.
64+
65+
Use the coverage summary to identify files with low or declining coverage and prioritize review attention on untested changes.
66+
67+
For more information about how the coverage percentage is calculated, see [AUTOTITLE](/code-security/reference/code-quality/metrics-and-ratings#code-coverage).
68+
5769
## Next steps
5870

5971
* Remediate quality findings in your default branch and improve the maintainability and reliability rating for your repository. See [AUTOTITLE](/code-security/code-quality/tutorials/improve-your-codebase).
Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
title: Setting up code coverage for your repository
3+
shortTitle: Set up code coverage
4+
intro: 'Upload test coverage reports to see coverage results directly on pull requests, helping reviewers identify untested code before merging.'
5+
versions:
6+
feature: code-quality
7+
product: '{% data reusables.gated-features.code-quality-availability %}'
8+
permissions: '{% data reusables.permissions.code-quality-repo-enable %}'
9+
contentType: how-tos
10+
layout: inline
11+
category:
12+
- Improve code quality
13+
---
14+
15+
{% data reusables.code-quality.code-quality-preview-note %}
16+
17+
In the following procedures, you will generate a Cobertura XML coverage report from your test suite, upload it to {% data variables.product.github %}, and view the coverage results on your pull requests.
18+
19+
## Prerequisites
20+
21+
* {% data variables.product.prodname_code_quality_short %} is enabled for your repository.
22+
* Your repository has a test suite that runs in {% data variables.product.prodname_actions %}.
23+
* Your test framework can produce a coverage report in **Cobertura XML** format.
24+
25+
## Step 1: Generate a Cobertura XML coverage report
26+
27+
Configure your test framework to output a coverage report in the Cobertura XML format. Code coverage works with any programming language that can produce this format.
28+
29+
1. Identify the coverage tool for your language from the table below.
30+
1. Add the appropriate command or configuration to your CI workflow so that a Cobertura XML file is generated each time your tests run.
31+
32+
| Language | Framework / Tool | How to generate Cobertura XML |
33+
|----------|-----------------|-------------------------------|
34+
| Python | `pytest` + `pytest-cov` | `pytest --cov=. --cov-report=xml` |
35+
| Java | JaCoCo | Use the `cover2cover.py` script or the JaCoCo-to-Cobertura Gradle/Maven plugin |
36+
| JavaScript/TypeScript | Istanbul / `nyc` | `nyc report --reporter=cobertura` |
37+
| Ruby | SimpleCov | Add `SimpleCov::Formatter::CoberturaFormatter` |
38+
| Go | `go test` + `gocover-cobertura` | `go test -coverprofile=cover.out && gocover-cobertura < cover.out > coverage.xml` |
39+
40+
> [!TIP]
41+
> If your framework isn't listed above, check its documentation for Cobertura output support. Many tools either support it directly or can convert to Cobertura XML from other formats.
42+
43+
## Step 2: Upload the coverage report
44+
45+
After your tests generate a Cobertura XML report, upload it to {% data variables.product.github %} so coverage results appear on pull requests.
46+
47+
1. Open your repository's CI workflow file (for example, `.github/workflows/ci.yml`).
48+
1. Add the following step after the step that runs your tests and generates the coverage report:
49+
50+
```yaml copy
51+
- name: Upload coverage report
52+
if: {% raw %}github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository{% endraw %}
53+
uses: actions/upload-code-coverage@v1
54+
with:
55+
report: COVERAGE-FILE-PATH.xml
56+
language: LANGUAGE
57+
label: LABEL
58+
```
59+
60+
1. Replace the following values:
61+
* **`COVERAGE-FILE-PATH.xml`**: The path to your Cobertura XML report (for example, `coverage.xml` or `target/site/jacoco/cobertura.xml`).
62+
* **`LANGUAGE`**: The primary language of the code being covered (for example, `Python`, `Java`, `JavaScript`).
63+
* **`LABEL`**: An optional label to identify this coverage report (for example, `code-coverage/pytest`).
64+
1. Commit and push the workflow change.
65+
66+
### Full workflow example
67+
68+
This example runs Python tests with `pytest-cov` and uploads the coverage report:
69+
70+
```yaml annotate copy
71+
# This workflow runs your test suite, generates a Cobertura XML coverage report, and uploads it to {% data variables.product.github %}. Once this workflow is committed, coverage results appear automatically on every pull request.
72+
name: Code Coverage
73+
74+
# Run on pushes to the default branch (to establish the baseline) and on pull requests (to compare against it). {% data variables.product.prodname_code_quality_short %} compares PR branch coverage to the default branch, so both triggers are needed.
75+
on:
76+
push:
77+
branches: [main]
78+
pull_request:
79+
branches: [main]
80+
81+
# The `code-quality: write` permission is required to upload coverage data. No other elevated permissions are needed.
82+
permissions:
83+
contents: read
84+
code-quality: write
85+
86+
jobs:
87+
test:
88+
runs-on: ubuntu-latest
89+
steps:
90+
# Check out the PR head commit (not the merge commit) so coverage line numbers map correctly to the diff.
91+
- uses: {% data reusables.actions.action-checkout %}
92+
with:
93+
ref: {% raw %}${{ github.event.pull_request.head.sha || github.sha }}{% endraw %}
94+
95+
# Replace this step with whatever language setup your project uses (Node.js, Java, Go, etc.). The upload action works with any language that produces a Cobertura XML report.
96+
- uses: {% data reusables.actions.action-setup-python %}
97+
with:
98+
python-version: "3.x"
99+
100+
- name: Install dependencies
101+
run: |
102+
python -m pip install --upgrade pip
103+
pip install -r requirements.txt
104+
pip install pytest pytest-cov
105+
106+
# Adapt this step for your test framework. The key requirement is producing a Cobertura XML file. For other languages, see the framework table earlier in this article.
107+
- name: Run tests with coverage
108+
run: pytest --cov=. --cov-report=xml
109+
110+
# This step replaces any third-party coverage upload (Codecov, Coveralls, etc.). After this runs, the `{% data variables.code-quality.pr_commenter %}` bot posts a coverage summary directly on the pull request.
111+
- name: Upload coverage report
112+
if: {% raw %}github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository{% endraw %}
113+
uses: actions/upload-code-coverage@v1
114+
with:
115+
report: coverage.xml
116+
language: Python
117+
label: code-coverage/pytest
118+
```
119+
120+
## Step 3: View coverage results on pull requests
121+
122+
1. Open a pull request (or push to an existing one) that triggers the workflow you configured.
123+
1. After the workflow completes, look for a comment from `{% data variables.code-quality.pr_commenter %}` on the pull request. The comment includes:
124+
* The aggregate coverage percentage for the pull request branch compared to the default branch.
125+
* A per-file breakdown showing which files gained or lost coverage.
126+
127+
## Next steps
128+
129+
* **Interpret results:** Understand coverage metrics and per-file breakdowns on your pull requests. See [AUTOTITLE](/code-security/how-tos/maintain-quality-code/interpret-results).

content/code-security/reference/code-quality/metrics-and-ratings.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,22 @@ These ratings are used to summarize the overall reliability and maintainability
4747
| **Fair** | Codebase has moderate-severity issues that may impact quality, but are not critical. | ≥1 "Warning" level finding |
4848
| **Needs Improvement**| Codebase has high-severity issues, including bugs or major maintainability risks. | ≥1 "Error" level finding |
4949

50+
## Code coverage
51+
52+
Code coverage measures what percentage of your source code is executed when your test suite runs. {% data variables.product.prodname_code_quality_short %} displays a coverage percentage on pull requests after you upload a Cobertura XML coverage report.
53+
54+
### How coverage is calculated
55+
56+
The coverage percentage represents the number of lines covered by tests divided by the total number of lines, expressed as a percentage. {% data variables.product.prodname_code_quality_short %} stores the latest upload for each branch (including the default branch) and compares the PR branch coverage to the default branch coverage.
57+
58+
For example, if your default branch has 44% coverage and your PR branch has 65% coverage, the PR gained 21 percentage points of coverage.
59+
60+
### Per-file delta
61+
62+
The per-file breakdown on pull requests shows how coverage changed for each modified file. A positive delta means the file gained coverage on the PR branch compared to the default branch.
63+
64+
To set up code coverage for your repository, see [AUTOTITLE](/code-security/how-tos/maintain-quality-code/set-up-code-coverage).
65+
5066
## Further reading
5167

5268
* [AUTOTITLE](/code-security/code-quality/concepts/about-code-quality)

content/copilot/concepts/agents/cloud-agent/about-cloud-agent.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,7 @@ You can customize {% data variables.copilot.copilot_cloud_agent %} in a number o
148148
* **{% data variables.product.prodname_copilot_short %} can only make changes in the repository specified when you start a task**. {% data variables.product.prodname_copilot_short %} cannot make changes across multiple repositories in one run.
149149
* **By default, {% data variables.product.prodname_copilot_short %} can only access context in the repository specified when you start a task**. The {% data variables.product.prodname_copilot_short %} MCP server is configured by default to allow {% data variables.product.prodname_copilot_short %} to access context (for example issues and historic pull requests) in the repository where it is working. You can, however, configure broader access. See [AUTOTITLE](/copilot/how-tos/use-copilot-agents/cloud-agent/extend-cloud-agent-with-mcp).
150150
* **{% data variables.product.prodname_copilot_short %} can only work on one branch at a time** and can open exactly one pull request to address each task it is assigned.
151+
* **Each {% data variables.copilot.copilot_cloud_agent %} session has a maximum execution time of 59 minutes**. This is a hard limit that cannot be extended or bypassed. If a task exceeds this limit, the session will time out and stop. For complex tasks that may require more time, consider breaking the work into smaller, more focused tasks. You can configure a shorter timeout using the `timeout-minutes` setting in your `copilot-setup-steps.yml` file. See [AUTOTITLE](/copilot/how-tos/use-copilot-agents/cloud-agent/customize-the-agent-environment).
151152

152153
### Limitations in {% data variables.copilot.copilot_cloud_agent %}'s compatibility with other features
153154

0 commit comments

Comments
 (0)