Commit c929178
committed
feat: add the shard types online shard split routes by
A collection's shards are fixed at creation today. Online shard split
changes that count without taking the collection offline, and this is the
contract the pieces of it agree on: what a shard is, which keys it owns, and
how a row's routing key is computed.
schema.proto gains ShardState, the four states a shard passes through while
a split runs; CollectionShardInfo, one shard's state, vchannel and routing
predicate; and HashRouting, the residues of the collection's modulus that
shard owns. CollectionShardInfo supersedes the single-field
etcdpb.CollectionShardInfo the truncate API left in the collection meta,
keeping its field number so records already persisted still decode -- their
missing fields read as a normal, never-split shard, which is what they are.
DescribeCollectionResponse gains shard_infos, index-parallel to
virtual_channel_names; shard_by, the expression producing a row's routing
value; and routing_modulus, the one modulus the whole collection's residues
are taken against. Its shards_num keeps its number and changes meaning: it
is the count of shards serving now, which a split moves, rather than the
CreateCollection argument.
common.proto gains MsgType SplitShard, numbered with the collection DDL it
belongs to rather than the WAL group it travels through.
Placement is by modulus throughout. Range routing and a RoutingMode enum
were considered and dropped: with the comparison key reduced to the routing
value itself, ranges bought nothing hash buckets do not, and one scheme
needs no enum to select it. The oneof keeps a single member so a second
scheme has somewhere to arrive.
issue: #619
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>1 parent fa96f8c commit c929178
7 files changed
Lines changed: 6099 additions & 5436 deletions
File tree
- go-api
- commonpb
- milvuspb
- schemapb
- proto
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments