Skip to content

Dynamic fields are not returned by output_fields=['*'] or [''] in Milvus Lite 3.1.0 #347

Description

@zc277584121

Summary

Milvus Lite 3.1.0 does not return dynamic fields when searching with output_fields=["*"] or output_fields=["$meta"].

The same repro returns dynamic fields on Milvus Standalone and Zilliz Cloud. This makes client code behave differently across Lite, server, and cloud when it reconnects to an existing dynamic-field collection and does not know all dynamic field names ahead of time.

Environment

  • pymilvus==3.0.0
  • milvus-lite==3.1.0
  • Milvus Lite server version reported by get_server_version(): milvus_lite-3.1.0

Comparison backends tested:

Backend Version reported output_fields=["*"] output_fields=["$meta"]
Milvus Lite milvus_lite-3.1.0 only pk only pk
Milvus Standalone 2.5.13 includes dynamic fields includes dynamic fields
Milvus Standalone 3.0-beta includes dynamic fields includes dynamic fields
Zilliz Cloud Compatible with Milvus 2.6 includes dynamic fields includes dynamic fields

Reproduction

import os
import tempfile
from pymilvus import DataType, MilvusClient

uri = os.path.join(tempfile.mkdtemp(), "milvus.db")
client = MilvusClient(uri=uri)
print(client.get_server_version())

collection_name = "dynamic_output_repro"

schema = client.create_schema(auto_id=True, enable_dynamic_field=True)
schema.add_field(field_name="pk", datatype=DataType.INT64, is_primary=True, auto_id=True)
schema.add_field(field_name="text", datatype=DataType.VARCHAR, max_length=1024)
schema.add_field(field_name="vector", datatype=DataType.FLOAT_VECTOR, dim=2)

index_params = client.prepare_index_params()
index_params.add_index(field_name="vector", index_type="AUTOINDEX", metric_type="L2")

client.create_collection(
    collection_name=collection_name,
    schema=schema,
    index_params=index_params,
    consistency_level="Strong",
)
client.insert(
    collection_name=collection_name,
    data=[{"text": "foo", "vector": [0.0, 0.0], "page": 1, "source": "test"}],
)
client.flush(collection_name)
client.load_collection(collection_name)

for fields in [["*"], ["$meta"], ["pk", "text", "$meta"], ["pk", "text", "page", "source"]]:
    result = client.search(
        collection_name=collection_name,
        data=[[0.0, 0.0]],
        anns_field="vector",
        search_params={"metric_type": "L2", "params": {}},
        limit=1,
        output_fields=fields,
    )
    print(fields, result[0][0].get("entity"))

Actual output on Milvus Lite 3.1.0

milvus_lite-3.1.0
['*'] {'pk': 1}
['$meta'] {'pk': 1}
['pk', 'text', '$meta'] {'pk': 1, 'text': 'foo'}
['pk', 'text', 'page', 'source'] {'pk': 1, 'text': 'foo', 'page': 1, 'source': 'test'}

Expected behavior

Milvus Lite should align with Milvus Standalone and Zilliz Cloud:

  • output_fields=["*"] should return dynamic fields such as page and source.
  • output_fields=["$meta"] should return dynamic fields without requiring the client to know each dynamic key ahead of time.

Explicitly listing dynamic field names works in Milvus Lite, so the data is stored and queryable. The issue appears specific to wildcard/meta output field expansion.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions