Skip to content

Scale to large at once could cause error #305

Description

@KevinZonda

This exception is happened when a large set of db is loaded parallelly (even cross several process), will cause

2025-07-17 00:25:55,844 [ERROR][handler]: RPC error: [load_collection], <MilvusException: (code=2000, message=Assert "init_flag_ == true"  => Mmap manager has not been init. at /workspace/milvus-lite/thirdparty/milvus/internal/core/src/storage/MmapManager.h:96
: segcore error)>, <Time:{'RPC start': '2025-07-17 00:25:55.842868', 'RPC error': '2025-07-17 00:25:55.844493'}> (decorators.py:140)
2025-07-17 00:25:55,844 [ERROR][load_collection]: Failed to load collection: naive_rag (milvus_client.py:953)
[ERROR] Failed to load db chunk /home/jovyan/rds/Imperial-FYP-PubMed-Preprocessing/largeChunks_content_NaiveRAG_0.6B/chunk4.db: <MilvusException: (code=2000, message=Assert "init_flag_ == true"  => Mmap manager has not been init. at /workspace/milvus-lite/thirdparty/milvus/internal/core/src/storage/MmapManager.h:96
: segcore error)>
[ERROR] PANIC!
Traceback (most recent call last):
  File "/home/jovyan/rds/Imperial-FYP-PubMed-Preprocessing/AtomicRAG/db/vecdb/milvus_server.py", line 75, in <module>
    raise e
  File "/home/jovyan/rds/Imperial-FYP-PubMed-Preprocessing/AtomicRAG/db/vecdb/milvus_server.py", line 71, in <module>
    db.load_collection()
  File "/home/jovyan/rds/Imperial-FYP-PubMed-Preprocessing/AtomicRAG/db/vecdb/milvus.py", line 25, in load_collection
    self.client.load_collection(collection_name=self.collection_name)
  File "/opt/conda/lib/python3.12/site-packages/pymilvus/milvus_client/milvus_client.py", line 954, in load_collection
    raise ex from ex
  File "/opt/conda/lib/python3.12/site-packages/pymilvus/milvus_client/milvus_client.py", line 951, in load_collection
    conn.load_collection(collection_name, timeout=timeout, **kwargs)
  File "/opt/conda/lib/python3.12/site-packages/pymilvus/decorators.py", line 141, in handler
    raise e from e
  File "/opt/conda/lib/python3.12/site-packages/pymilvus/decorators.py", line 137, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/pymilvus/decorators.py", line 173, in handler
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/pymilvus/decorators.py", line 116, in handler
    raise e from e
  File "/opt/conda/lib/python3.12/site-packages/pymilvus/decorators.py", line 86, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/pymilvus/client/grpc_handler.py", line 1277, in load_collection
    check_status(response)
  File "/opt/conda/lib/python3.12/site-packages/pymilvus/client/utils.py", line 64, in check_status
    raise MilvusException(status.code, status.reason, status.error_code)
pymilvus.exceptions.MilvusException: <MilvusException: (code=2000, message=Assert "init_flag_ == true"  => Mmap manager has not been init. at /workspace/milvus-lite/thirdparty/milvus/internal/core/src/storage/MmapManager.h:96

My test is when over >38, it will be caused. It is caused ramdomly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions