tunneld: Reach remote and federated tunnelds over the /connect bridge - #1896
Merged
Conversation
The feature was stale by design: --uds only ever moved the HTTP control channel onto a unix socket, while the data plane - the RSD dials to the reported tunnel-address:tunnel-port - always relied on TCP, so it never delivered a pure-unix path. - remote tunneld loses --uds; TunneldRunner binds TCP only - TunneldAddress narrows to (host, port); the UDS client transport (_UnixHTTPConnection) is gone - a --tunnel value with a non-numeric :suffix now fails with a clear BadParameter naming the removal, instead of being treated as a socket path
pymobiledevice3 now consumes the WS /connect endpoint itself, so every
command and library consumer works against a tunneld whose tunnel
interface is unreachable from the client - tunneld in a different docker
network stack, on another host, or behind an SSH port-forward.
- TunneldConnectDialer: an asyncio.open_connection-compatible dialer
(the seam RemoteServiceDiscoveryService already exposes for the
userspace tunnel) that bridges each device-bound connection through
/connect. The caller gets one end of a socket.socketpair whose other
end is pumped against the websocket - a real socket, so
ServiceConnection's get_extra_info('socket') contract holds. Each
pump ends when either leg closes, so closing the RSD tears down
every bridge; nothing needs explicit teardown.
- get_tunneld_devices()/get_tunneld_device_by_udid() take
bridge=None|bool: None bridges automatically when the tunneld host is
non-loopback; True forces it for addresses that only look local
(SSH port-forwards).
- --tunnel accepts UDID@HOST[:PORT] (IPv6 in brackets), which forces
the bridge; a tunneld that predates /connect fails with a clear
upgrade message.
Verified end-to-end against a real tunneld server (uvicorn/wsproto)
bridging into a local TCP echo server, including through
ServiceConnection.create_using_tcp.
Tunnelds that register each other (A -> B -> A) recursed on every GET /: the nesting spawned a growing tree of in-flight requests against the bounded to_thread executor, and the innermost fetches lost the race against their parents' 2s timeout - so a cycle dropped the very listings it was meant to merge. With two mutually registered tunnelds, each one's listing was missing the other's device. Federated requests now carry x-tunneld-hops-remaining (default 4), decremented per hop and not forwarded at zero, and entries reported through several paths are listed once. Verified with two real federated tunneld servers: the peer's device is now present (it was missing before) and each tunnel appears once.
Federating another tunneld only merged its listing, and the addresses in
that listing are ULAs on the upstream host's utun - reachable only from a
client that already had L3 connectivity to that host. So federation
presupposed a VPN or hand-rolled routes, which is why a device discovered
through an aggregator was a dead end.
The /connect bridge removes that prerequisite. A /connect for a device
none of this instance's own tunnels serve is now relayed to the upstream
that owns it, so a client needs a route to one tunneld and nothing else:
sudo pymobiledevice3 remote tunneld --host 0.0.0.0 \
--upstream http://lab-1:49151 --upstream http://lab-2:49151
pymobiledevice3 developer dvt ls / --tunnel 'UDID@front:49151'
- new --upstream (repeatable) seeds the set POST /upstream manages
- GET / entries carry an origin: null when served directly, else the URL
of the hop that reported them, so a client that can reach that upstream
may skip the relay
- the upstream's own close code is passed through, so 4404/4502 name the
tunneld that actually refused
- the /connect client is now shared (tunneld/ws_bridge.py) between the
dialer and this relay rather than hand-rolled twice
Verified against a real device: an aggregator holding no tunnels of its
own fronted a separate tunneld process and served dvt ls / and
device-information over the two-hop path, with no tracebacks on either
instance.
Federation reads an upstream address twice - requests fetches the listing from the URL, while the /connect relay dials the host and port parsed out of it - and the two disagreed on everything except the exact spelling 'http://HOST:PORT': - a schemeless 'lab-1:49151' (the spelling --tunnel UDID@HOST:PORT invites) parses as scheme 'lab-1' with no host: the listing fetch threw InvalidSchema into a debug log and the relay refused, so registering one did nothing at all, silently - an https:// upstream fetched its listing over TLS but was relayed to in plaintext, on port 80 when no port was given - a URL carrying a path fetched that path but relayed to the bare host Registration now runs both spellings through one normalizer, which fills in tunneld's default port, brackets IPv6 literals, and rejects what neither path could act on (non-http scheme, no host, a path or query, a bad port). --upstream reports the failure as a usage error and POST /upstream as a 400, instead of accepting an address that quietly never works. HOST, HOST:PORT and http://HOST[:PORT] are all accepted and stored canonically, so GET /upstream and DELETE /upstream agree.
A device can have more than one tunnel: when a tunneld federates another
that monitors the same host, its listing carries both its own tunnel and
the federated one for that UDID. /connect keyed on the UDID alone, so it
paired whichever tunnel it found locally with the port the caller asked
for - dialing one tunnel's address at another's port, an endpoint that
exists nowhere:
4502 failed to connect to [fdc9:ec7b:4fdf::1]:63727: [Errno 61]
Connect call failed
/connect now takes an optional ?address=, the tunnel address the caller
means. A local tunnel is used only when it matches; otherwise the request
falls through to the upstream that serves that exact tunnel, so holding
our own tunnel to a device no longer hides another instance's. The dialer
names the tunnel it is bridging for, and the relay passes it upstream.
Older tunnelds ignore the parameter and keep their UDID-only behaviour.
Docs: a front tunneld on a host that also has devices attached should run
with the monitors off, or it federates a device it is also serving and
lists it twice.
Verified against a real device in that exact topology: an aggregator
holding its own tunnel plus a federated one now serves dvt ls / over
either, where both previously failed.
doronz88
force-pushed
the
feature/tunneld-remote-bridge
branch
from
August 31, 2026 07:35
8087cfd to
44ba610
Compare
The test held a socket bound but not listening and connected to its port, expecting an RST. macOS does not send one - it drops the SYN silently - so tunneld burned its whole CONNECT_TCP_TIMEOUT before answering 4502, and on the slower macOS runners the test client's own recv timeout fired first: 'socket.timeout: timed out' on py3.9/macos-latest. Connect to a released port instead, which does get an RST. The file drops from 9.3s to 1.5s as a side effect. A port stolen in the gap makes the assertion fail loudly rather than pass silently.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #1893:
/connectexposed the tunnel over HTTP, but nothing in pymobiledevice3 consumedit, so a remote
tunneldwas only reachable by writing your own websocket client. This makes theclient speak it, and makes a
tunneldable to front othertunnelds over the same bridge.What
Reach a remote tunneld (
--tunnel UDID@HOST[:PORT]).TunneldConnectDialeris anasyncio.open_connection-compatible dialer plugged into the seamRemoteServiceDiscoveryServicealready exposes for the userspace tunnel, so there is no new call site: every device connection an
RSD opens is bridged through
/connect. The caller is handed one end of asocket.socketpairwhose other end is pumped against the websocket, so consumers that reach for
writer.get_extra_info('socket')(ServiceConnection) keep working. Bridging is automatic whenthe
tunneldhost is non-loopback, andbridge=True(the@form on the CLI) forces it foraddresses that only look local, such as an SSH port-forward.
Front several hosts with one tunneld (
--upstream, repeatable). Federation previously onlymerged an upstream's listing, and the addresses in that listing are ULAs on the upstream host's
utun— reachable only from a client that already had L3 connectivity to that host. So itpresupposed a VPN or hand-rolled routes, and a device discovered through an aggregator was a dead
end. A
/connectfor a device the instance does not serve itself is now relayed to the upstreamthat owns it:
GET /entries now carry anorigin—nullwhen served directly, else the URL of the hop thatreported them — so a client that can reach that upstream may skip the relay. Upstream close codes
are passed through, so a 4404/4502 names the
tunneldthat actually refused.Remove
--uds. It only ever moved the HTTP control channel onto a unix socket while the dataplane (the RSD dials, and now the bridge) always relied on TCP, so it never delivered a pure-unix
path.
TunneldAddressnarrows to(host, port); a--tunnelvalue with a non-numeric:suffixnow fails with a message naming the removal instead of being read as a socket path.
Bugs found and fixed along the way
other recursed on every
GET /: the nesting spawned a growing tree of in-flight requests againstthe bounded
to_threadexecutor, and the innermost fetches lost the race against their parents'2s timeout. With two mutually registered tunnelds, each one's listing was missing the other's
device. Requests now carry a hop budget (
x-tunneld-hops-remaining, 4), and a tunnel reportedthrough several paths is listed once.
/connectaddressed a device, not a tunnel. A device can have more than one tunnel — a fronttunneldthat also monitors the same host holds its own alongside the federated one. Keying onthe UDID alone paired whichever tunnel was found locally with the port the caller asked for,
dialing an endpoint that exists nowhere (
4502 failed to connect to [fdc9:…]:63727)./connectnow takes an optional
?address=; older tunnelds ignore it and keep their UDID-only behaviour.lab-1:49151parses as schemelab-1with nohost, so the listing fetch threw
InvalidSchemainto a debug log and the relay refused —registering one did nothing at all.
https://fetched over TLS but relayed in plaintext to port80. Registration now normalizes
HOST,HOST:PORTandhttp://HOST[:PORT]to one canonicalform and rejects what neither path could act on, as a usage error on
--upstreamand a 400 fromPOST /upstream.syslog live) left thepump coroutines to GC, which closes them with
GeneratorExit, where the awaits in theirfinallyblocks raised
coroutine ignored GeneratorExit. Teardown is synchronous now.Verification
Against a real iPhone (iOS 26.6.1) over USB, not only fakes:
dvt ls /anddvt device-informationthrough the bridge (--tunnel UDID@127.0.0.1, and theempty-UDID
@hostform), with zero tracebacks in the tunneld log.separate tunneld process — a topology that is self-proving, since it has no other way to reach
the device.
?address=fix in the failing topology: an aggregator holding both its own tunnel and afederated one to the same device now serves
dvt ls /over either, where both previously failed.Automated: 704 tests pass (
pytest -m "not device"), including new suites for the bridge(
tests/test_tunneld_remote_bridge.py) and federation (tests/test_tunneld_federation.py) thatspin up real
tunneldservers over uvicorn/wsproto against a TCP echo server. The cycle and?address=regressions were both confirmed to fail before their fixes.pyright --venvpath .(1.1.411) reports 0 errors;
ruff check/ruff format --check(0.15.4) and markdownlint are clean.Notes
/connectremains unauthenticated like the rest of thetunneldAPI, and an aggregator relaysinto other hosts' devices, which widens that exposure. The auth work offered in tunneld: Add /connect websocket endpoint exposing the tunnel over HTTP #1893 should
land before this topology is advertised broadly; the docs carry the warning meanwhile.
traffic, measurable on bulk transfers such as DDI mounts.
not listening gets silence rather than an RST, so
test_connect_tcp_connection_refusedburns thefull ~7.8s connect timeout on every run there.