Don't let a fixture failure blank out the greenboard upload - #515
Don't let a fixture failure blank out the greenboard upload#515torcolvin wants to merge 2 commits into
Conversation
Scope the __overall_fail guard to the in-process counter, which really is truncated once the flag latches. JUnit-XML counts are complete and have their own skip rules, so a run with 197 passes and one flaky fixture now publishes a red bar instead of nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
The updated overall-failure guard can still publish partial results if only one of pass_count/fail_count is overridden, and one newly added comment is misleading about when markers are observed.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR refines the greenboard upload behavior so that a setup/teardown (“overall”) failure only suppresses uploads when results rely on the in-process pytest hook counters (which can be truncated), while still allowing uploads when pass/fail counts are supplied externally (e.g., derived from JUnit XML). This fits the TDK harness by ensuring CI publishes a meaningful red/green signal instead of dropping the upload entirely for flaky fixture failures.
Changes:
- Adjust
GreenboardUploader.upload()to apply the overall-failure skip only when counts are not explicitly provided. - Ensure SGW marker detection still happens even when
__overall_failhas latched (so platform selection isn’t lost). - Add unit tests covering the revised overall-failure guard behavior and JUnit-XML-derived counting/skip rules.
File summaries
| File | Description |
|---|---|
client/tests/test_greenboarduploader.py |
Adds tests for overall-failure guard semantics and upload_from_junit_file JUnit-derived count behavior. |
client/src/cbltest/greenboarduploader.py |
Scopes the overall-failure guard to in-process counters and reorders marker tracking ahead of the short-circuit. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Scope the __overall_fail guard to the in-process counter, which really is truncated once the flag latches. JUnit-XML counts are complete and have their own skip rules, so a run with 197 passes and one flaky fixture now publishes a red bar instead of nothing.