Spike: dnsserver-quic (DoT, DoH, DoQ, DoH3 listeners and authenticated PROXYv2)

Spike (i) from architecture §14, plus the half of spike (b) that does not need dnsdist: the authenticated PROXYv2 parser. Relevant design: §3.3 (internal/dnsserver), §5.2 (ports), §6.2 (listener config), D45, D56, D67.

Verdict: pass. All four TLS transports answer correctly through the shared pipeline, checked with an independent client (dnsproxy upstream) and with stock tools (dig, curl, dnsproxy CLI). Every PROXYv2 forgery case is dropped and counted on UDP, TCP and DoT. The binary-size delta is +996 KiB (+8.6 %), of which quic-go accounts for 448 KiB.

Environment for every number here: Linux amd64, 8 vCPU (Go reports GOMAXPROCS 6 because of the cgroup quota), Go 1.27.2, quic-go v0.63.0, miekg/dns v1.1.73, dnsproxy v0.86.0. Client and server ran on loopback in the same VM, so the figures compare transports with each other; they are not capacity claims.

What was built

All new code sits under internal/dnsserver, plus the config keys (internal/config, which the brief allowed).

cmd/kudurru is untouched, as the brief required. Wiring the registry into serve takes four lines: SpecsFromConfig, PolicyFromConfig, NewRegistry, and Run as a runnable feeding checker. That is a follow-up for whoever owns cmd/.

Decisions taken in the spike (please confirm or record)

1. PROXYv2 headers are optional, never trusted by default. A datagram or connection without the signature is an ordinary query attributed to its transport peer, even from an allowed peer. That keeps the loopback readiness self-query and dig @127.0.0.1 working. A header is honoured only if the peer is listed and the header is authenticated. Any other header is dropped and counted. PowerDNS's proxy-protocol-from instead requires a header from listed peers. Requiring it adds no security, because a header-less query can only claim the peer's own address.

2. "Loopback" means exactly 127.0.0.1 and ::1, as D56 lists them. 127.0.0.2 is treated as an ordinary peer: it needs the secret, which is also what makes the rule testable on one host. Loopback must still appear in proxyProtocolFrom. The exemption only waives the TLV, and only while no secret is set.

3. Exact hosts only. NewPolicy refuses 10.0.0.0/24, fd00::/64 and anything else wider than /32 or /128 (D45/D56). See the surprise about §5.2 below.

4. Secret format. The TLV value is compared in constant time with the secret file's bytes, after trailing CR/LF are trimmed. The secret must be at least 32 bytes. openssl rand -hex 32 (64 printable bytes, 32 bytes of entropy) is the recommended form, because Helm randAlphaNum and dnsdist's string parameter then carry exactly the same bytes, with no binary-escaping question.

5. Drop reasons (the bounded label values for kudurru_dns_dropped_total{reason=…}):

   • proxy_unauthenticated: the peer is allowed but the TLV is missing or wrong.
   • proxy_untrusted_peer: the peer is not listed.
   • proxy_malformed: the header is truncated, out of spec, longer than 2 KiB, or uses AF_UNIX.

   LOCAL commands and AF_UNSPEC headers still need authentication, and then keep the peer address.

6. PROXYv2 applies to the udp, tcp and dot listeners. DoT costs nothing extra because the header precedes TLS on the same TCP wrapper. DoH, DoQ and DoH3 do not parse headers, because dnsdist talks Do53 to backends. With no proxyProtocolFrom configured nothing is inspected, so the miekg *net.UDPConn fast path is kept.

7. Fresh DNS IDs. DoQ queries (ID 0 by RFC 9250) and DoH/DoH3 queries (ID 0 recommended by RFC 8484) enter the pipeline with a crypto-random ID. The client's ID is restored on the response, so no stage can carry the client's fixed ID onto an upstream wire (the CVE-2026-47703 class). The upstream stage also randomises IDs. TestDoQFreshIDsPerQuery sends 32 queries and checks that the upstream saw at least 28 distinct IDs, none of them 0.

8. DoH body cap is 65,535 bytes, not 2 MiB. No DNS message can be larger than 65,535 bytes, so a 2 MiB cap would buffer 32 times more per in-flight request for no benefit. A larger body gets 413, an oversized ?dns= gets 414, headers are capped at 16 KiB, and all of these are configurable through Limits. This deviates from the brief; please confirm.

9. DoH3 reports pipeline.ProtoDoH, because pipeline.Proto has no HTTP/3 value and internal/pipeline was out of scope. The registry key and readiness name are still doh3. See the recommendation below.

10. QUIC limits follow CoreDNS: 256 bidirectional streams per connection, 1024 DoQ stream workers per listener, 30 s idle timeout, 0-RTT allowed (queries are replay-safe, and the ingress stage refuses non-QUERY opcodes). DoQ grants no unidirectional streams. Limits.QUICMaxConns is implemented (excess connections get DOQ_EXCESSIVE_LOAD or H3_EXCESSIVE_LOAD) but defaults to unlimited, whereas CoreDNS defaults to 200. A config knob should expose it.

11. DoQ error policy. Every RFC 9250 §4.3.3 violation closes the connection with DOQ_PROTOCOL_ERROR, as CoreDNS and dnsproxy do: zero-length prefix (draft framing), FIN before the declared length, data after the query, no FIN before the 2 s read deadline, non-zero ID, edns-tcp-keepalive, or an unpackable message.

Measurements

1. Binary size (CGO_ENABLED=0 -trimpath -ldflags '-s -w', as in the Makefile)

git archive --format=tar -o /tmp/kudurru-base.tar 79a0e37   # the scaffold this spike started from
git archive --format=tar -o /tmp/kudurru-head.tar HEAD
# extract both; in a third copy of HEAD delete internal/dnsserver/doq.go and doh3.go and the
# ProtoDoQ/ProtoDoH3 lines of the factories table ("noquic")
CGO_ENABLED=0 go -C /tmp/kudurru-base   build -trimpath -ldflags '-s -w' -o /tmp/ksize/kudurru-base   ./cmd/kudurru
CGO_ENABLED=0 go -C /tmp/kudurru-noquic build -trimpath -ldflags '-s -w' -o /tmp/ksize/kudurru-noquic ./cmd/kudurru
CGO_ENABLED=0 go -C /tmp/kudurru-head   build -trimpath -ldflags '-s -w' -o /tmp/ksize/kudurru-head   ./cmd/kudurru
CGO_ENABLED=0 go -C /tmp/kudurru-head   build -trimpath -ldflags '-s -w' -tags spike          -o /tmp/ksize/spikecmd          ./internal/dnsserver/spikecmd
CGO_ENABLED=0 go -C /tmp/kudurru-head   build -trimpath -ldflags '-s -w' -tags spike,dnsproxy -o /tmp/ksize/spikecmd-dnsproxy ./internal/dnsserver/spikecmd

cmd/kudurru is unchanged, yet it already carries everything. The factory table is initialised at package init, so the linker keeps every listener once internal/dnsserver is imported. For the non-QUIC +548 KiB, a go tool nm -size diff of unstripped builds points mostly at the stdlib TLS server handshake (crypto/tls +98 KB of symbols), ECDSA/RSA signing, x509 private-key parsing, golang.org/x/crypto and x/text, and the HTTP/2 server. The scaffold linked net/http but never served TLS, so the linker had dropped those parts.

dnsproxy/upstream is a test-only dependency today, so it adds 0 bytes to kudurru. It brings five modules into the build list: AdguardTeam/dnscrypt, AdguardTeam/golibs and ameshkov/dnsstamps (all Unlicense), robfig/cron/v3 (MIT) and golang.org/x/exp. None is GPL or EUPL. Its go.mod pins quic-go v0.60.0; MVS selects v0.63.0 and dnsproxy builds and passes against it. The weight is reasonable, and §2 already plans to link it for internal/upstream, which will cost the +1.07 MiB measured above.

2. Handshake + first query on loopback

Each sample opens a new connection with a fresh tls.Config (full handshake, no resumption), sends one A query, then times a second query on the same connection. The pipeline is a local-answer stage, so transport cost dominates. n = 200 per protocol; the table shows the median of three runs, with the range in brackets.

KUDURRU_SPIKE_MEASURE=1 go test -run 'TestSpikeMeasure$' -count=1 -v ./internal/dnsserver/

The TLS transports cost about 1 ms of CPU-bound handshake on loopback (ECDSA P-256, X25519). After the handshake, DoT is as cheap as TCP, and DoQ is cheaper per query than either HTTP transport. One DoH3 run had a 10.6 ms p99 outlier. It is a scheduling artefact of sharing CPUs with the client; the other two runs measured 1.9 and 2.3 ms.

3. Queries per second, 4 connections, 5 s per run

Same harness and local-answer stage. "x 1" means each connection has one query in flight (closed loop). "x 16" means 16 concurrent queries per connection, which only the multiplexing transports support (miekg's client cannot pipeline). The table shows the median of three runs, with the range in brackets. There were zero errors in every run after the MaxTCPQueries fix (see Surprises).

Even with client and server sharing six cores, every transport is far above what a home or office network generates. DoQ with multiplexing roughly matches Do53/TCP. DoH3 is the most expensive transport per query (HTTP/3 framing and QPACK on top of QUIC; not profiled further). That is acceptable.

4. Goroutines at 256 concurrent DoQ streams

The DoQ-only server runs in a child process, so client goroutines are not counted. A gate stage parks every query until the target count has arrived; the server then samples runtime.NumGoroutine() and its heap.

KUDURRU_SPIKE_MEASURE=1 go test -run 'TestSpikeDoQGoroutines$' -count=1 -v ./internal/dnsserver/

DoQ server, 1 conn(s) x 256 streams: goroutines before any connection 11, with 1 idle connection(s) 14, at 256 in-flight queries: goroutines 270, heap in use 3.1 MiB; failed exchanges 0
DoQ server, 4 conn(s) x 256 streams: goroutines before any connection 11, with 4 idle connection(s) 23, at 1024 in-flight queries: goroutines 1047, heap in use 6.5 MiB; failed exchanges 0

Each idle QUIC connection costs 3 goroutines: two inside quic-go plus our stream acceptor. Each in-flight stream costs exactly one goroutine (the worker), so 256 streams give 270 goroutines and about 3 MiB of heap. The 1024-worker cap bounds the total at about 1050 goroutines per listener whatever the connection count. Further streams wait in QUIC flow control: TestDoQStreamAndWorkerLimits holds 4 workers against 40 streams, and checks that stream 257 on one connection gets StreamLimitReachedError.

5. PROXYv2 parser cost and fuzzing

go test -run XXX -bench . -benchtime 2s ./internal/dnsserver/proxyproto/
BenchmarkParse-6      	138267898	        17.31 ns/op	       0 B/op	       0 allocs/op
BenchmarkDatagram-6   	26900391	        89.37 ns/op	       0 B/op	       0 allocs/op

Datagram covers the full per-packet work: signature check, peer lookup, parse and constant-time comparison of a 64-byte secret. It adds 89 ns and no allocations to each proxied UDP query.

Each fuzz target ran for 60 s, seeded with the golden header, IPv4/IPv6/LOCAL/UNIX headers and plain DNS bytes:

go test -run XXX -fuzz '^FuzzParse$'    -fuzztime 60s ./internal/dnsserver/proxyproto/
go test -run XXX -fuzz '^FuzzDatagram$' -fuzztime 60s ./internal/dnsserver/proxyproto/
go test -run XXX -fuzz '^FuzzStream$'   -fuzztime 60s ./internal/dnsserver/proxyproto/

6. Test suite

go test -race -count=1 ./internal/dnsserver/... ./internal/config/   # all green, about 4 s for dnsserver
golangci-lint run ./...                                              # 0 issues (also with --build-tags spike,dnsproxy)
CGO_ENABLED=0 go build ./... && go vet ./...                         # clean

Notable tests:

• TestTransportsEndToEnd: all six transports through ingress → recorder → upstream to a fake server, using dnsproxy upstream as the client.
• TestDoQProtocolErrors: seven RFC 9250 violations, wrong ALPN, and unidirectional streams.
• TestDoHHTTPSemantics: twelve status-code cases, h1.1 and h2, and the DoH3 path.
• TestSharedCertificateReload: rotates the certificate behind DoT, DoH, DoQ and DoH3, and checks that new handshakes see it without a restart.
• TestKubernetesSecretSwap: the ..data symlink swap.
• TestProxyProtocolEndToEnd: the forgery matrix on real UDP, TCP and DoT sockets, using 127.0.0.2/127.0.0.3 as peers.
• TestLongLivedStreamConnections, TestUDPLargeQueryNotTruncated.

Operator commands

Start every listener with a generated certificate. spikecmd prints the certificate path; pass -cert/-key for your own pair, and -upstream 1.1.1.1:53 to forward instead of answering A 192.0.2.1:

go run -tags spike ./internal/dnsserver/spikecmd   # udp/tcp :5300, dot :8853, doh :8443, doq :8853/udp, doh3 :8443/udp
go run -tags spike,dnsproxy ./internal/dnsserver/spikecmd -selfcheck   # queries all six through dnsproxy/upstream and exits

Every command below was run against spikecmd (script /tmp/kspike/smoke.sh during the spike) and returned 192.0.2.1, except the kdig lines. kdig is not installed in the spike VM; those lines use the documented Knot 3.3+ syntax and are untested here.

Running PROXYv2 against dnsdist (untested here, because the VM has no dnsdist binary):

openssl rand -hex 32 > /run/secrets/proxy
go run -tags spike ./internal/dnsserver/spikecmd -dns 127.0.0.1:5353 -proxy-from 127.0.0.1 -proxy-secret-file /run/secrets/proxy
# dnsdist.conf:
#   newServer({address="127.0.0.1:5353", useProxyProtocol=true})
#   addAction(AllRule(), SetAdditionalProxyProtocolValueAction(0xE0, "<contents of /run/secrets/proxy>"))

For the metrics owner: Registry.Proxy() returns the *proxyproto.Policy. Register kudurru_dns_dropped_total{reason} either as a CounterFunc per proxyproto.Reasons entry reading policy.Dropped(reason) (lock-free atomics), or by incrementing a CounterVec from policy.SetDropHook.

Surprises

1. The worktree branch started at the initial commit (5a893a4), not at the scaffold (79a0e37) that the other spike worktrees use. I fast-forwarded the branch to 79a0e37 before starting. No rebase was needed.
2. miekg/dns closes TCP and DoT connections after 128 queries (MaxTCPQueries default). The first throughput run showed DoT and TCP stalling at about 100 q/s with millions of errors. The fix (commit c13bb60) defaults to unlimited, as CoreDNS does (tcpMaxQueries = -1), bounded by the 10 s idle timeout. The scaffold's Server had the same latent bug.
3. miekg/dns reads UDP into a 512-byte buffer by default (UDPSize). An EDNS query larger than 512 bytes was silently truncated and dropped, and a PROXYv2 header (16 + 12 + 67 bytes with the TLV) eats into that budget. The UDP listeners now use 4096 bytes, plus proxyproto.MaxHeaderLen when PROXYv2 is on. TestUDPLargeQueryNotTruncated fails at the old setting.
4. §5.2 contradicts D56. It says --proxy-protocol-from=<edge pod CIDR>, while D56 and this implementation accept exact /32s only. In Kubernetes, edge pods have dynamic IPs, so a hostOnly, restart-required list of exact peers cannot track them.
5. Linking is all-or-nothing. Because the factory table is a package-level map, kudurru carries quic-go (448 KiB) even before cmd/ wires DoQ. That is fine under D67, but nothing can be dropped from a "small" build without a build tag.
6. dnsproxy's go.mod pins quic-go v0.60.0 while CoreDNS and blocky pin v0.63.0. MVS picks v0.63.0 and dnsproxy compiles and passes against it. This needs re-checking on every quic-go bump, because quic-go is pre-1.0 and changes its API between minor versions.
7. Wrapping the UDP socket for PROXYv2 loses miekg's *net.UDPConn OOB path, which picks the reply source address on multi-homed hosts bound to 0.0.0.0. That is harmless for dnsdist on loopback or in-pod, but a listener on a multi-homed host with PROXYv2 enabled should bind a specific address. The wrapper is only installed when proxyProtocolFrom is non-empty.
8. Go's TLS server refuses mismatched ALPN. A DoT client offering only h2 gets no_application_protocol. Clients offering no ALPN, or dot, work. This is the correct behaviour for DoT, and the operator docs should mention it.

Recommended changes to docs/architecture.md (not edited here)

1. §5.2 / D56: resolve the pod-CIDR contradiction. Either (a) allow CIDR entries in proxyProtocolFrom only when proxyProtocolSecretFile is set, so the TLV carries the authentication and the CIDR only narrows exposure, or (b) keep exact /32s and say how Kubernetes supplies them. The spike recommends (a). It is a one-line change in proxyproto.parseHost, and D45's rationale (CIDR trust alone is forgeable) still holds.
2. §3.3: record decisions 1, 2, 4 and 5 above: optional headers, loopback meaning exactly 127.0.0.1 and ::1, secret bytes equal to the file contents, and the three bounded drop reasons.
3. §3.3 / pipeline.Proto: add ProtoDoH3 (label doh3), so metrics and the query log can tell HTTP/3 from HTTP/2. Until then DoH3 reports doh.
4. §3.3 limits: record MaxTCPQueries unlimited with a 10 s idle timeout, the UDP read buffer of 4096 + PROXYv2 headroom, the DoH body cap of 65,535 bytes rather than 2 MiB, and the QUIC connection cap (implemented, default unlimited, needs a config key).
5. §11.1: add kudurru_dns_dropped_total{reason} with the reason values listed above.
6. §6.2: mention that the dot, doh, doq and doh3 listeners share one reloader per cert/key pair, with fsnotify plus a 30 s poll fallback, and that a half-rotated pair keeps the old certificate.