Repository navigation
feat: use MixTransport streams for DHT proxy lookups - #1581
Draft
marcinczenko wants to merge 1 commit into
Draft
marcinczenko wants to merge 1 commit into
marcinczenko wants to merge 1 commit into
Conversation
Share the existing Mix transport with proxy discovery and terminate lookup streams at the proxy. Require matching Mix addresses and use protocol v2 with explicit message limits instead of the single-packet reply budget. Update the relay tool, bound exchanges and cleanup, and cover multi-packet responses, failures, cancellation, and the absence of direct DHT fallback. Refs: logos-storage/logos-storage-pm#41
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
🤖 GPT-6 via Codex drafting on behalf of @marcinczenko
Refs logos-storage/logos-storage-pm#41, part of Use Mix Transport for DHT Proxies — #34.
DHT proxy lookups currently use a single-reply Mix connection: a separate exit connects to the proxy, reads its response through a registered destination-read callback, and returns it using one SURB. That limits provider responses to one Mix packet.
This first iteration moves the exchange onto a reliable MixTransport stream terminating at the proxy itself (
exit == destination). The proxy performs the direct DHT lookup on behalf of the client and sends the provider records back over that same stream. MixTransport supplies segmentation, reliable delivery, and reply SURBs.Implementation
MixPeerTransportand shared MixTransport lifecycle. Each lookup owns a stream; closing it leaves the shared session available to downloads and subsequent lookups./mixaddress whose embedded peer identity matches the record. The address supplies the proxy's Mix public information. Ordinary TCP addresses alone are insufficient.MixProtocol.toConnectionexchange and the DHT proxy destination-read registrations.mix_relay_dhtproxy mode, advertise Mix public information in its signed peer records, and stop the transport during shutdown.Protocol and rollout
The codec changes to
/storage/dht-proxy/2.0.0. Protobuf fields and status values stay the same, with explicit limits of 4 KiB per request and 256 KiB per response, independent of Mix packet capacity. Provider lists are still trimmed to the largest prefix fitting the response limit; a first record that cannot fit producesErrResponseTooLarge.Version 1 proxies cannot serve this flow. Upgrade proxies and distribute their updated signed peer records before switching clients. There is no legacy-protocol fallback. Dynamic proxy discovery remains tracked in #32. Related work: Service Discovery — PR #1582 adds discovery for DHT Mix proxies and other services. Integration with this PR's MixTransport-based proxy exchange remains to be assessed and validated. Follow-up: Integrate service discovery with MixTransport DHT proxies — PM #45.
The existing
storage.nimbleandnimble.lockdependency selections are unchanged.Validation
Validated locally with Nim 2.2.14 and the locked dependencies:
tests/storage/testdhtproxyclient.nim: 8 passed. Includes a real five-node Mix network with a stub DHT backend, a 100-provider response larger than the previous packet budget, subsequent lookups, malformed requests, capacity errors, timeout, cancellation, proxy failure, address/identity checks, response packing limits, and no direct fallback.tests/storage/blockexchange/discovery/testdiscovery.nim: 5 passed.nim c --compileOnly storage.nim: passed.nim c --compileOnly -d:mixVersion:pm41 tools/mix/mix_relay_dht.nim: passed.nph;git diff --checkpassed.523ce010is GPG-signed and verified locally and by GitHub.Draft review scope
This is a first iteration for review, not a claim that all of #41's validation is complete. The full repository suite, live DHT/fleet rollout, simultaneous proxy lookup and download traffic, and broader loss/load testing have not been run for this change. Review the wire-version migration, chosen limits, and shared-session cleanup before promotion from draft. Remote CI is separate from the local evidence above.