Skip to content

chore(worker): extract the Tokio data-plane path into tokio_conn - #355

Merged
beinan merged 2 commits into
milvus-io:mainfrom
beinan:split-worker-main
Jul 28, 2026
Merged

chore(worker): extract the Tokio data-plane path into tokio_conn#355
beinan merged 2 commits into
milvus-io:mainfrom
beinan:split-worker-main

Conversation

@beinan

@beinan beinan commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Refs #306. First of the splits proposed there — the one with a real seam.

Why this file, this cut

crates/talon-worker/src/main.rs was 1509 lines covering CLI parsing, backend construction, io_uring capability probing, ring supervision, coordinator registration, heartbeat, signal handling, and the entire Tokio data-plane connection handler.

The connection handler is the piece that clearly does not belong there, and the codebase already says so. uring_conn.rs is a library module whose own docs open with:

This is the completion-based counterpart to the Tokio handle_conn in main.rs.

So two implementations of one wire protocol lived at different layers — one in the library, one inline in a binary. They must stay behaviourally identical (same frames, same limits, same error semantics), and that is much easier to review and to keep honest when they sit side by side as siblings.

Change

Moved to a new crates/talon-worker/src/tokio_conn.rs:

  • handle_conn, handle_control_frame, handle_put, handle_delete, sendfile_payload, read_control

Only handle_conn and read_control are pub — the other four stay private so the library's public API does not widen.

main.rs: 1509 → 1059 lines.

The module doc explains why this path pays for sendfile (Tokio's TcpStream is non-blocking, so a blocking sendfile would spuriously EAGAIN, forcing an into_std/from_std round-trip on every transfer) and points at uring_conn for the contrast, mirroring the existing cross-reference in the other direction.

This is a pure move

The function bodies are byte-identical. Verified mechanically:

$ diff <(git show HEAD:.../main.rs | sed -n '617,1066p') <(tail -n +38 .../tokio_conn.rs)

differs only in the doc-comment line offset and the trailing newline. The remaining diff is imports: removed from main.rs what moved out, added to tokio_conn.rs what it needs, and pushed four now-test-only imports into the mod tests block.

No behaviour change, so the existing suite is the regression test.

A gap this exposed

Worth flagging, because it is the kind of thing #306 argued splitting would surface. With the path in its own file, coverage attributes it separately:

uring_conn.rs   78.68%   <- the default since #299
tokio_conn.rs   36.41%   <- the fallback
main.rs         55.04%

The two paths are supposed to be behaviourally identical, so 36% on the fallback is a real gap — it was previously averaged into main.rs and invisible. Not addressed in this PR; this change only makes it measurable. Worth its own issue.

Verification

$ cargo build --workspace --all-features    # no warnings
$ cargo clippy --workspace --all-targets --all-features -- -D warnings   # clean
$ cargo test --workspace --all-features     # 36 suites, all ok

Not included

The other splits in #306 (runtime.rs, coordinator/observability.rs) are separate mechanical changes and belong in separate PRs. I have deliberately left talon-core/src/config.rs alone — as noted in the issue, it is the source of truth for the generated configuration reference and splitting it risks the config-docs drift gate for little gain.


Update: PR title changed refactor:chore: (the repo's conventional-commits gate allows feat|fix|enhance|test|doc|bench|build|ci|chore, not refactor), and fixed a self-inflicted docs failure: the module docs linked to sendfile_payload, which this PR makes private, tripping -D rustdoc::private_intra_doc_links.

beinan added 2 commits July 28, 2026 05:42
`main.rs` was 1509 lines spanning CLI parsing, backend construction, io_uring
capability probing, ring supervision, registration, heartbeat, signal handling
*and* the whole Tokio data-plane connection handler.

The connection handler is the part with a natural seam. `uring_conn.rs` was
already a library module, and its own docs describe it as "the completion-based
counterpart to the Tokio `handle_conn` in `main.rs`" — so the two
implementations of one protocol lived at different layers, one in the library
and one inline in a binary. This makes them structurally parallel siblings.

Moved verbatim (bodies are byte-identical; `git diff` shows moves and imports
only): handle_conn, handle_control_frame, handle_put, handle_delete,
sendfile_payload, read_control. Only handle_conn and read_control are `pub`;
the rest stay private so the library's public API does not widen.

main.rs 1509 -> 1059 lines.

Measured effect worth noting: with the path in its own file, coverage now
attributes it separately, and the Tokio fallback sits at 36.4% line coverage
against uring_conn's 78.7%. Both paths must stay behaviourally identical, so
that gap is a real gap — it was previously hidden inside main.rs's aggregate.
Not addressed here; this change only makes it visible.

Refs milvus-io#306.
The tokio_conn module docs linked to `sendfile_payload`, which is private,
tripping `-D rustdoc::private_intra_doc_links` in the docs gate.
@beinan beinan changed the title refactor(worker): extract the Tokio data-plane path into tokio_conn build(worker): extract the Tokio data-plane path into tokio_conn Jul 28, 2026
@beinan beinan changed the title build(worker): extract the Tokio data-plane path into tokio_conn chore(worker): extract the Tokio data-plane path into tokio_conn Jul 28, 2026
@beinan
beinan merged commit 50c4dd8 into milvus-io:main Jul 28, 2026
22 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant