Skip to content

feat: use MixTransport streams for DHT proxy lookups - #1581

Draft
marcinczenko wants to merge 1 commit into
masterfrom
feat/dht-proxy-mix-transport
Draft

marcinczenko wants to merge 1 commit into
masterfrom
feat/dht-proxy-mix-transport

Conversation

@marcinczenko

@marcinczenko marcinczenko commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Note

🤖 GPT-6 via Codex drafting on behalf of @marcinczenko

Refs logos-storage/logos-storage-pm#41, part of Use Mix Transport for DHT Proxies — #34.

DHT proxy lookups currently use a single-reply Mix connection: a separate exit connects to the proxy, reads its response through a registered destination-read callback, and returns it using one SURB. That limits provider responses to one Mix packet.

This first iteration moves the exchange onto a reliable MixTransport stream terminating at the proxy itself (exit == destination). The proxy performs the direct DHT lookup on behalf of the client and sends the provider records back over that same stream. MixTransport supplies segmentation, reliable delivery, and reply SURBs.

Implementation

  • Reuse Storage's existing MixPeerTransport and shared MixTransport lifecycle. Each lookup owns a stream; closing it leaves the shared session available to downloads and subsequent lookups.
  • Require a proxy signed peer record with a valid /mix address whose embedded peer identity matches the record. The address supplies the proxy's Mix public information. Ordinary TCP addresses alone are insufficient.
  • Remove the legacy MixProtocol.toConnection exchange and the DHT proxy destination-read registrations.
  • Bound the entire client attempt, including session establishment and stream opening, to 90 seconds. Timeout and cancellation cancel the exchange and close its stream. Discovery continues trying configured proxies on failure and never falls back to a direct client-side DHT lookup.
  • Count incoming exchanges against the proxy's capacity before reading requests. Keep the five-second request-read limit and add a 90-second server exchange deadline.
  • Start MixTransport in the standalone mix_relay_dht proxy mode, advertise Mix public information in its signed peer records, and stop the transport during shutdown.
  • Document the flow, resource limits, configuration, and rollout in docs/dht-proxy.md.

Protocol and rollout

The codec changes to /storage/dht-proxy/2.0.0. Protobuf fields and status values stay the same, with explicit limits of 4 KiB per request and 256 KiB per response, independent of Mix packet capacity. Provider lists are still trimmed to the largest prefix fitting the response limit; a first record that cannot fit produces ErrResponseTooLarge.

Version 1 proxies cannot serve this flow. Upgrade proxies and distribute their updated signed peer records before switching clients. There is no legacy-protocol fallback. Dynamic proxy discovery remains tracked in #32. Related work: Service Discovery — PR #1582 adds discovery for DHT Mix proxies and other services. Integration with this PR's MixTransport-based proxy exchange remains to be assessed and validated. Follow-up: Integrate service discovery with MixTransport DHT proxies — PM #45.

The existing storage.nimble and nimble.lock dependency selections are unchanged.

Validation

Validated locally with Nim 2.2.14 and the locked dependencies:

  • tests/storage/testdhtproxyclient.nim: 8 passed. Includes a real five-node Mix network with a stub DHT backend, a 100-provider response larger than the previous packet budget, subsequent lookups, malformed requests, capacity errors, timeout, cancellation, proxy failure, address/identity checks, response packing limits, and no direct fallback.
  • tests/storage/blockexchange/discovery/testdiscovery.nim: 5 passed.
  • nim c --compileOnly storage.nim: passed.
  • nim c --compileOnly -d:mixVersion:pm41 tools/mix/mix_relay_dht.nim: passed.
  • Changed Nim files formatted with nph; git diff --check passed.
  • Commit 523ce010 is GPG-signed and verified locally and by GitHub.

Draft review scope

This is a first iteration for review, not a claim that all of #41's validation is complete. The full repository suite, live DHT/fleet rollout, simultaneous proxy lookup and download traffic, and broader loss/load testing have not been run for this change. Review the wire-version migration, chosen limits, and shared-session cleanup before promotion from draft. Remote CI is separate from the local evidence above.

Share the existing Mix transport with proxy discovery and terminate lookup
streams at the proxy. Require matching Mix addresses and use protocol v2
with explicit message limits instead of the single-packet reply budget.

Update the relay tool, bound exchanges and cleanup, and cover multi-packet
responses, failures, cancellation, and the absence of direct DHT fallback.

Refs: logos-storage/logos-storage-pm#41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant