fix(bulk_writer): propagate sync commit failures - #3780
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: SiluPanda The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Welcome @SiluPanda! It looks like this is your first PR to milvus-io/pymilvus 🎉 |
|
Tick the box to add this pull request to the merge queue (same as
|
|
Closing this for now: this repository enforces DCO sign-off, and I cannot provide or maintain that legal attestation for this contribution. The patch remains available on the branch for maintainers or a human contributor to reuse if helpful. |
8758421 to
9cf4df2
Compare
Summary
Fixes #2314.
This change makes synchronous bulk-writer commits propagate failures from the background flush thread instead of only logging them and returning normally.
The current implementation starts
_flush()on a thread, joins it for the default synchronouscommit(), and then resets the buffer state. Ifpersist()or a remote upload callback fails, the exception stays inside the worker thread, so callers see a successful return and may continue with an emptybatch_filesresult.The patch records the first flush exception and re-raises it on the caller thread:
This keeps the existing asynchronous flush model, but makes the default synchronous
LocalBulkWriter.commit()andRemoteBulkWriter.commit()fail loudly and become catchable by usertry/exceptblocks.This also covers the same root cause reported in #2628 for
RemoteBulkWriter.commit().Testing
pytest -q tests/unit/test_local_bulk_writer.py tests/unit/test_remote_bulk_writer.pymake lint