Separated timeouts + cleaner handling - #485
Conversation
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR improves the SGW QE Jenkins pipeline’s reliability around timeouts by separating infra setup from test execution and ensuring pytest can exit gracefully to preserve artifacts/log collection.
Changes:
- Split Jenkins pipeline into separate “Infra Setup” and “Run Tests” stages with distinct timeouts.
- Add pytest session and per-test timeouts, plus a SIGTERM→SIGINT pytest plugin to allow graceful teardown under Jenkins termination.
- Enhance sgcollect behavior/documentation to collect on session timeout as well as test failures.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| jenkins/pipelines/QE/sgw/test.sh | Adds --skip-setup, guards setup execution, and introduces pytest timeouts for graceful shutdown. |
| jenkins/pipelines/QE/sgw/Jenkinsfile | Splits infra setup vs test execution into nested stages with separate timeout budgets. |
| client/src/cbltest/plugins/sigterm_handler.py | New pytest plugin to convert SIGTERM into SIGINT so pytest runs cleanup/finalizers. |
| client/src/cbltest/plugins/sgcollect_fixture.py | Collect sgcollect artifacts on session timeout as well as on failures; updates help text. |
| client/pyproject.toml | Adds pytest-timeout dependency and registers the new pytest plugin entrypoint. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
From some reason this page is unable to load merge status and parent branch-PR for this PR, hence lets hold on this. |
Co-authored-by: Tor Colvin <tor.colvin@couchbase.com>
Co-authored-by: Tor Colvin <tor.colvin@couchbase.com>
|
I feel this PR advances to a cleaner approach for Jenkins testing, hence I request we keep it alive, why I think so :
Please take into account, the tech debt that is being created for SGW pipeline causing the current test-runtime-bloat difficult to be answerable for. |
|
I don't mind the separated steps but it's cleaner and easier to read if you separate the two steps into two scripts rather than having one script do both with differing flags. |
The stages in Jenkinsfile for infra-setup and test-run have been separated so that the confusion between how much time is a total
test.shis taking (which has both infra setup and thenuv run) can be cleaner, simpler and granular.This also helps now in allocating better timeouts for each of the two stages, hence the total time-taken for the test run will be now more precise.
COMPARED TO the previous ballpark measure of 2hours (inclusive of infra+run)
MORE IMPORTANTLY : This is also better in terms of helping out with a bug which might be related to increased INFRA time vs increased TEST-RUN time, since if the infra time is increased, Jenkins would timeout right there, it won't waste the time running the tests as well, leading to harder debugging later on.