Skip to content

fix: check_invalid_binary_vector should not raise on non-sized values - #3769

Open
lokeshramchand-ctrl wants to merge 2 commits into
milvus-io:masterfrom
lokeshramchand-ctrl:fix/check-invalid-binary-vector-non-sized-values
Open

fix: check_invalid_binary_vector should not raise on non-sized values#3769
lokeshramchand-ctrl wants to merge 2 commits into
milvus-io:masterfrom
lokeshramchand-ctrl:fix/check-invalid-binary-vector-non-sized-values

Conversation

@lokeshramchand-ctrl

Copy link
Copy Markdown

Problem

check_invalid_binary_vector raises an unhandled TypeError instead of returning False when a BINARY_VECTOR entity's values list contains an element with no __len__ (e.g. None, an int):

>>> check_invalid_binary_vector([{"type": DataType.BINARY_VECTOR, "values": [b"\x00\x01", None]}])
TypeError: object of type 'NoneType' has no len()

The per-row loop calls len(values) before checking isinstance(values, bytes):

for values in entity["values"]:
    if len(values) * 8 != dim:        # len() evaluated first
        return False
    if not isinstance(values, bytes):  # too late — never reached on non-sized input
        return False

Impact

This function's whole job is to return False so the caller can raise a clean error. grpc_handler.py:1094 (and :1228, plus the async handler) does:

if not check_invalid_binary_vector(entities):
    raise ParamError(message="Invalid binary vector data exists")

So instead of ParamError("Invalid binary vector data exists"), the user gets a raw TypeError out of the validation helper. Reachable from a malformed binary-vector insert, e.g. Collection.insert(data=[[b"\x00\x01", None]]).

This is a sibling gap to the one fixed in #3694 (andor for the empty-list case), in the same function, that survived that fix. The existing test_non_bytes_values test only exercises nested lists as the invalid case, which still have __len__, so it never caught a genuinely non-sized element.

Filed as #3768.

Fix

Swap the two checks: verify isinstance(values, bytes) before calling len(values).

for values in entity["values"]:
    if not isinstance(values, bytes):
        return False
    if len(values) * 8 != dim:
        return False

Testing

Added test_non_sized_value to TestCheckInvalidBinaryVector in tests/unit/test_utils.py, alongside the existing test_inconsistent_dimensions / test_non_bytes_values / test_empty_values cases.

  • uv run pytest tests/unit/test_utils.py -q → 71 passed
  • uv run pytest tests/unit/ -q → 4719 passed, 3 skipped, 1 failed (tests/unit/test_check.py::TestGetCommit::test_get_commit, a pre-existing unrelated failure — same one called out in Reject an empty binary vector list instead of raising IndexError #3694's own PR description — verified to fail identically on master before this change)
  • Manually verified the repro above raises TypeError before the fix and returns False after

This change was AI-assisted (Claude). I found the bug while doing a broader audit for unclaimed defects, reproduced the TypeError directly, traced the call sites in grpc_handler.py / async_grpc_handler.py to confirm the intended contract, and ran the full unit suite plus the baseline comparison above myself.

@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: lokeshramchand-ctrl
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot

Copy link
Copy Markdown

Welcome @lokeshramchand-ctrl! It looks like this is your first PR to milvus-io/pymilvus 🎉

@lokeshramchand-ctrl

Copy link
Copy Markdown
Author

/assign @XuanYang-cn -- Please look into this and let me know if any changes

Comment thread pymilvus/client/utils.py
Comment thread tests/unit/test_utils.py
lokeshramchand-ctrl and others added 2 commits August 27, 2026 17:53
check_invalid_binary_vector called len(values) before isinstance(values,
bytes), so a BINARY_VECTOR row containing a non-bytes, non-sized element
(e.g. None) raised a raw TypeError instead of returning False. Callers in
grpc_handler.py and async_grpc_handler.py rely on False to raise a clean
ParamError("Invalid binary vector data exists"), so this crashed cleanly
malformed inserts with a confusing TypeError instead.

Swap the isinstance check ahead of the len() check. Fixes milvus-io#3768.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: lokeshramchand-ctrl <lokeshramchand@gmail.com>
yhmo pointed out that check_invalid_binary_vector still raised
TypeError when the *first* element of values was non-bytes, since
dim was computed from entity["values"][0] before the per-element
isinstance check ran. Guard the first element before computing dim,
and add regression tests for a non-sized first element and an
all-non-sized list.

Signed-off-by: lokeshramchand-ctrl <lokeshramchand@gmail.com>
@lokeshramchand-ctrl
lokeshramchand-ctrl force-pushed the fix/check-invalid-binary-vector-non-sized-values branch from 132ee01 to 8900f65 Compare August 27, 2026 17:54
@mergify mergify Bot added dco-passed and removed needs-dco labels Aug 27, 2026
@mergify

mergify Bot commented Aug 27, 2026

Copy link
Copy Markdown

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants