Dynamic array field filters fail in Milvus Lite 3.1.0

[zc277584121]

Summary

Milvus Lite 3.1.0 can store and return dynamic array fields, and dynamic scalar filters work, but filtering on a dynamic array field using array indexing fails with unknown field.

The same array filter works on Milvus Standalone and Zilliz Cloud.

Environment

• pymilvus==3.0.0
• milvus-lite==3.1.0
• Milvus Lite server version reported by get_server_version(): milvus_lite-3.1.0

Comparison backends tested:

Backend	Version reported	rank == 1	topic == "a"	array_field[0] < 1
Milvus Lite	milvus_lite-3.1.0	works	works	fails with unknown field 'array_field'
Milvus Standalone	2.5.13	works	works	works
Milvus Standalone	3.0-beta	works	works	works
Zilliz Cloud	Compatible with Milvus 2.6	works	works	works

Reproduction

import os
import tempfile
from pymilvus import DataType, MilvusClient

uri = os.path.join(tempfile.mkdtemp(), "milvus.db")
client = MilvusClient(uri=uri)
print(client.get_server_version())

collection_name = "dynamic_array_filter_repro"

schema = client.create_schema(auto_id=True, enable_dynamic_field=True)
schema.add_field(field_name="pk", datatype=DataType.INT64, is_primary=True, auto_id=True)
schema.add_field(field_name="text", datatype=DataType.VARCHAR, max_length=1024)
schema.add_field(field_name="vector", datatype=DataType.FLOAT_VECTOR, dim=2)

index_params = client.prepare_index_params()
index_params.add_index(field_name="vector", index_type="AUTOINDEX", metric_type="L2")

client.create_collection(
    collection_name=collection_name,
    schema=schema,
    index_params=index_params,
    consistency_level="Strong",
)
client.insert(
    collection_name=collection_name,
    data=[
        {"text": "foo", "vector": [0.0, 0.0], "rank": 1, "topic": "a", "array_field": [0, 1, 2]},
        {"text": "bar", "vector": [1.0, 1.0], "rank": 2, "topic": "b", "array_field": [1, 2, 3]},
    ],
)
client.flush(collection_name)
client.load_collection(collection_name)

for expr in ["rank == 1", 'topic == "a"', "array_field[0] < 1"]:
    result = client.search(
        collection_name=collection_name,
        data=[[0.0, 0.0]],
        anns_field="vector",
        search_params={"metric_type": "L2", "params": {}},
        limit=2,
        filter=expr,
        output_fields=["pk", "text", "rank", "topic", "array_field"],
    )
    print(expr, [hit["entity"] for hit in result[0]])

Actual output on Milvus Lite 3.1.0

milvus_lite-3.1.0
rank == 1 [{'pk': 1, 'text': 'foo', 'rank': 1, 'topic': 'a', 'array_field': [0, 1, 2]}]
topic == "a" [{'pk': 1, 'text': 'foo', 'rank': 1, 'topic': 'a', 'array_field': [0, 1, 2]}]
MilvusException: unknown field 'array_field' at column 1
  array_field[0] < 1
  ^^^^^^^^^^^
available fields: ['pk', 'text', 'vector']

Expected behavior

Since the dynamic array field is stored and returned correctly, and the same filter works on Milvus Standalone and Zilliz Cloud, Milvus Lite should support array_field[0] < 1 for dynamic array fields as well.