Skip to content

ci: probe whether GitHub runners can execute io_uring - #281

Merged
beinan merged 1 commit into
milvus-io:mainfrom
beinan:ci/278-iouring-probe
Jul 25, 2026
Merged

ci: probe whether GitHub runners can execute io_uring#281
beinan merged 1 commit into
milvus-io:mainfrom
beinan:ci/278-iouring-probe

Conversation

@beinan

@beinan beinan commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

What

Adds an informational CI job that answers one question with data: can GitHub-hosted runners actually execute io_uring?

No production code touched. The job is continue-on-error and never gates a merge.

Why

#273 measured a thread-per-core io_uring (monoio) data plane winning on all three of latency, per-core throughput, and memory (+35% throughput, −19% p50, −26% p99, −34% RSS vs full Tokio; 8.05× scaling at 8 rings). Before that migration lands, the test strategy depends entirely on whether CI can run io_uring:

  • If supported — the new data plane can be exercised directly in CI, same as the existing fuse mount e2e job runs a real kernel mount.
  • If not — it must be feature-gated, with CI compiling but not executing it, and correctness verified on self-hosted or production hardware.

That is a fork in the road for the migration PR, so it is worth resolving first rather than guessing.

Kernel version, the io_uring_disabled sysctl, and seccomp filtering are three independent gates — a container can satisfy the first two and still block the syscall — so this probes actual runtime behaviour instead of inferring it.

What it checks

  1. A monoio::RuntimeBuilder::<IoUringDriver> runtime builds (catches panics, since monoio panics rather than erroring when a blocking pool is missing).
  2. A SO_REUSEPORT listener binds and completes a real TCP roundtrip — this is the exact mechanism thread-per-core scaling depends on, not just "the syscall exists".
  3. bind_to_cpu_set works (CPU pinning, the other half of thread-per-core).

Plus the environment surface: kernel, arch, nproc, io_uring_disabled, seccomp mode, container detection. All of it lands in the job summary.

Isolation

The probe crate has its own manifest with an empty [workspace] stanza, so it is excluded from cargo build --workspace and pulls monoio into no normal build. Verified locally:

members: [talon-core, talon-coordinator, talon-transport, talon-worker,
          talon-backend, talon-fuse, talon-client, talon-observability]

.github/probe/target/ is gitignored. The workflow only triggers on workflow_dispatch or PRs touching the probe itself, talon-transport, or talon-worker.

Local result

On this dev host (kernel 6.6.141, io_uring_disabled=0, no seccomp):

URING_BUILD=ok
REUSEPORT_BIND=ok 127.0.0.1:42011
ROUNDTRIP=probe
CPU_PIN=ok
PROBE_RESULT=SUPPORTED

The point of this PR is to learn whether the same holds on ubuntu-latest.

Verification

  • cargo build --workspace --all-targets --all-features --locked — unaffected
  • cargo fmt --all --check — clean
  • Probe runs green locally via the committed manifest

Refs #273

🤖 Generated with Claude Code

DESIGN.md targets a thread-per-core io_uring (monoio) data plane, and milvus-io#273
measured it to win on latency, per-core throughput, and memory. Before any
migration lands we need to know whether CI can execute io_uring at all,
because that determines whether the new data plane can be tested here or must
be feature-gated and verified elsewhere.

Kernel support, the io_uring_disabled sysctl, and seccomp filtering are three
independent gates, and a container can satisfy the first two while still
blocking the syscall — so this probes actual runtime behaviour rather than
inferring it from the kernel version.

The probe reports three things: that a monoio IoUringDriver runtime builds,
that a SO_REUSEPORT listener binds and completes a real TCP roundtrip (the
mechanism thread-per-core scaling depends on), and that bind_to_cpu_set works.

The probe crate carries its own manifest with an empty [workspace] stanza so
it is excluded from `cargo build --workspace` and pulls monoio into no normal
build. The job is continue-on-error and writes its verdict to the job summary;
it never gates a merge, since its purpose is to answer a design question.

Refs milvus-io#273

Co-Authored-By: Claude <noreply@anthropic.com>
@beinan
beinan force-pushed the ci/278-iouring-probe branch from 33a9970 to 4708960 Compare July 25, 2026 09:36
@beinan
beinan merged commit eaa2b17 into milvus-io:main Jul 25, 2026
14 checks passed
beinan added a commit that referenced this pull request Jul 25, 2026
…286)

First step of #285. Adds transport::uring — the completion-based counterpart
to limits::read_frame — alongside the Tokio reader rather than replacing it.
Purely additive: no caller changes, so it lands and is tested in isolation.

The two cannot share an implementation. Tokio's AsyncRead borrows a buffer for
the duration of a read; io_uring is completion-based, so the kernel owns the
buffer while the operation is in flight and monoio's AsyncReadRent moves it in
and hands it back with the result. A borrowed-buffer signature cannot be made
sound over a completion ring.

What the wire format shares, the limits must share too. The DoS protections
from #111 are reimplemented verbatim and covered by tests that would fail if
they regressed: the advertised length is checked against the per-message-type
cap before the payload buffer is allocated (a control frame claiming a
data-plane-sized payload is rejected without committing that memory), both
reads are timeout-bounded so a peer stalling mid-frame is dropped rather than
pinning a buffer, a clean EOF at a frame boundary is Eof rather than an error,
and a truncated payload is an I/O error rather than a short frame that would
desync the protocol.

Ten tests run on a real io_uring runtime over loopback TCP, including
consecutive frames on one connection to prove the reader leaves the stream at
the next frame boundary. CI can execute these: the capability probe added in
#281 confirmed io_uring works on GitHub-hosted runners, which is why this
needs no feature gate.

monoio is added with the "sync" feature, which provides the spawn_blocking the
zero-copy sendfile path will need to stay off the ring. Note it requires an
explicitly attached thread pool and panics without one, unlike tokio's.

Refs #285, #273

Co-authored-by: Claude <noreply@anthropic.com>
@beinan
beinan deleted the ci/278-iouring-probe branch July 25, 2026 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant