What are Commit Statuses?
Commit statuses are indicators displayed on GitHub pull requests that show whether certain checks or tests have passed or failed. Qlty can add statuses based on your code coverage metrics, helping you enforce quality standards.Available Coverage Statuses
Qlty provides two types of coverage statuses:Total Coverage
This status shows the total code coverage percentage for your pull request, and its delta as compared to the base branch. You can configure it to fail if:- Total coverage decreases compared to the base branch
Diff Coverage
This status shows the code coverage percentage only for lines changed in the pull request. You can configure it to fail if:- Diff coverage is below a minimum threshold
Enabling Coverage Statuses
To enable coverage statuses:- Go to your project in Qlty
- Navigate to “Project Settings” > “Code Review”
- Turn on Qlty Coverage and/or Qlty Coverage diff
- Configure the thresholds according to your requirements
Configuration Options
Total Coverage Options
- Total Coverage Variation: If total coverage drops by more than this many percentage points, Qlty will post a failed status to this commit. Qlty compares the drop as the status shows it, rounded to one decimal place (or to as many as the setting has), so with a variation of 1%, a drop shown as -1.0% passes and a drop shown as -1.1% fails.
Diff Coverage Options
- Diff Coverage: The minimum coverage required for changed lines
- Minimum Diff Size: The minimum number of executable changed lines for the threshold to apply. Qlty counts only lines in files that have coverage data. Below this size the status still posts, showing the diff coverage percentage followed by “(Fewer than N lines)”, and it passes.
Example Configurations
Strict Coverage Policy
This configuration enforces high standards for both total and differential coverage:- Qlty Coverage:
- Total Coverage Variation: 1%
- Qlty Diff Coverage:
- Minimum diff coverage: 90%
- Minimum lines: 5
Gradual Improvement Policy
This configuration allows for gradual improvement while still maintaining standards:- Qlty Coverage:
- Total Coverage Variation: 1%
- Qlty Diff Coverage:
- Minimum diff coverage: 30%
- Minimum lines: 5
Branch Protection Rules
To fully enforce your coverage requirements, integrate these statuses with GitHub branch protection rules:- Go to your repository on GitHub
- Navigate to “Settings” > “Rules” > “Rulesets”
- Edit or create a branch ruleset
- Under “Rules” check “Require status checks to pass”, click “Add Checks” and search for and select:
qlty coverage(total coverage)qlty coverage diff(diff coverage)
- Save your changes
Troubleshooting
Status Not Appearing
If you don’t see the commit status:- Ensure your Qlty GitHub App has been granted access to your repo. (Head to Workspace Settings > Connections to configure permissions)
- Ensure that you’ve enabled the commit status from your Project Settings > Code Review page.
- Verify that a coverage report was successfully uploaded for the head commit of your PR. Qlty posts the statuses when that report finishes processing. (See Project Settings > Code Coverage for a list of uploaded reports).
- A missing report on the base branch doesn’t stop the statuses from posting, but the total coverage status can’t show a change without one. Publish coverage on every push to your base branch so pull requests have a report to compare against.
- Head to Troubleshooting Coverage for more info.
Status Always Failing
If your status is always failing:- Review your threshold settings and adjust if they’re too strict
- Check if your tests are running correctly and generating accurate coverage data
- Verify the coverage report is being uploaded correctly
Status Passing Without a Threshold Check
In these cases a status passes without checking your threshold, and its description says why:- Diff smaller than the Minimum Diff Size. The diff coverage status shows the percentage followed by “(Fewer than N lines)”.
- No coverage data for any of the changed files. The diff coverage status reads “Not applicable. There was no coverage data for the files in this diff.”
- No executable changed lines, for example when a pull request changes only comments or configuration. The diff coverage status reads “Not applicable. No executable lines in diff.”
- Diff too large to read. When GitHub can’t return the pull request’s diff, for example because it changes more than 500 files, the diff coverage status gives the reason.
- No base coverage to compare against. The total coverage status shows the head’s total followed by “(Unknown change: Coverage of base commit is unavailable)”. This happens when the base branch has no coverage report at or before the pull request’s base commit.
- Selective coverage. The total coverage status reads “Not computable for select coverage”. See Selective Coverage.
Best Practices
- Start with realistic thresholds: Begin with attainable goals and gradually increase them
- Apply different thresholds for different projects: New projects can aim for higher coverage than legacy codebases
- Use both total and diff coverage: Enforce high standards for new code while gradually improving overall coverage
See Also
- Quality Gates - Learn about other quality gates in Qlty
- PR Comments - Configure code coverage comments on pull requests
- Coverage Metrics - Understand the different coverage metrics Qlty provides