Research note, 2026-10-10. Point-in-time study produced during the kudurru design phase and cited by docs/architecture.md. It describes other projects for analysis only; nothing in it is to be ported into kudurru (several of them are GPL/EUPL). Items marked UNVERIFIED were not confirmed against a primary source. Verify before relying on a version number or API detail.

dnsdist + keepalived as kudurru components

Research key: dnsdist-keepalived. Date: 2026-10-10. Primary sources: shallow clones of PowerDNS/pdns (HEAD 3fe6ea9, 2026-10-09) and acassen/keepalived (HEAD 2d0b479, 2026-09-24) in /tmp/kudurru-research/dnsdist-keepalived/, plus dnsdist.org, keepalived.org, kube-vip.io, metallb.io, Docker Hub, GitHub API. Anything I could not confirm from a primary source is marked UNVERIFIED.

0. TL;DR for the architecture doc

Decisions this research supports:

1. Use dnsdist's YAML config (2.0+) as the generated artifact, plus a tiny Lua companion file that only defines functions. kudurru's controller renders dnsdist.yml from cluster state.
2. Do runtime membership changes through the encrypted console (dnsdist -e 'newServer(...)', rmServer(uuid)) or a Lua maintenance() loop that resolves a headless Service / kudurru's own member list, and only fall back to a restart for structural changes (binds, TLS, cache sizes). The REST API cannot add/remove backends; it is read-only except allow-from and cache flush.
3. keepalived belongs to the Docker/bare-metal topology. In Kubernetes prefer kube-vip (services mode, per-service election, externalTrafficPolicy: Local) or MetalLB; keep a keepalived DaemonSet option only for clusters where a VRRP VIP must coexist with non-k8s VRRP peers or where the operator already standardises on VRRP.
4. GPL boundary: both are invoked as separate executables over the network/CLI, never linked into the Apache-2.0 kudurru binary. Shipping them as sidecar images (or even re-packaging upstream binaries in a kudurru-published image with the GPL text) is fine; dynamic or static linking is not.

1. dnsdist

1.1 Identity, versions, licensing

• Source: pdns/pdns/dnsdistdist/ inside github.com/PowerDNS/pdns. License GPL-2.0 (repo COPYING; Docker Hub README: "licensed under GPLv2").
• Release train, from docs/changelog.rst in the clone (verified):
  • 2.0.10 — 22 Sep 2026 (bug fix: dynamic suffix blocking via FFI, regression from 2.0.9)
  • 2.1.2 — 8 Sep 2026 (YAML exception handling, DoQ/DoH3 response drops, Prometheus DoH3 latency, carbon host-name dots)
  • 2.1.1 / 2.0.8 / 1.9.16 — 6 Aug 2026
  • 2.1.0 — 2 Jul 2026 (OpenTelemetry tracing, structured logging default on, addServerStateChangeCallback, health-check response validator, AllowedRCodeRatio dynamic rule, h2o removed)
  • 2.0.0 — 21 Jul 2025 (first stable with YAML config)
  • 1.9.x is the legacy train (security fixes only; 1.9.12 2026-03-31 fixed CVE-2026-24029 DoH ACL skip)
  • dnsdist-2.2.0-alpha0 tag exists (upgrade guide already documents 2.1.x -> 2.2.0 changes).
• Upgrade-guide facts you must know when targeting 2.1+ (verified in docs/upgrade_guide.rst):
  • 2.1.0: default config file is now dnsdist.yml (falls back to dnsdist.conf); a dnsdist.lua next to it is auto-loaded without configuration directives; h2o removed (nghttp2 only, so DoH is HTTP/2-only — no HTTP/1.1 behind a non-h2 reverse proxy); structured logging on by default; webserver no longer allows cross-origin requests; Lua LB policies must return an index.
  • 2.2.0 (upcoming): webserver binds IPV6_V6ONLY (listen on 0.0.0.0 and [::] separately); queries with TC set are dropped; AXFR/IXFR over UDP answered "Not Implemented" immediately; ClientState.muted removed; new newServer options ecdheCurves, maxOutstandingQueries.
• Threading model (docs/advanced/tuning.rst, verified): one UDP thread + one TCP acceptor thread per bind; TCP worker pool (setMaxTCPClientThreads, default 10); one UDP responder thread per backend; one maintenance thread (calls Lua maintenance() every second, cleans cache); one health-check thread; console thread + one per connection; webserver threads + one per HTTP connection; optional carbon/SNMP threads. Scale UDP with multiple identical binds using reusePort (YAML binds[].reuseport: true, threads: N).

1.2 Configuration formats: Lua vs YAML (status as of 2026)

• Lua (dnsdist.conf) is the original format and remains fully supported in 2.x.
• YAML (dnsdist.yml) exists since 2.0.0; 2.1.0 made it the default file name. The authoritative schema is pdns/dnsdistdist/dnsdist-settings-definitions.yml (2,486 lines; Rust bindings are generated from it — building with YAML needs rustc/cargo and Python+PyYAML).
• Rules of coexistence (docs/reference/yaml-settings + upgrade guide):
  • A -C file.yml is YAML; dnsdist also loads a sibling .lua file but only for functions (no configuration directives) unless enableLuaConfiguration() is called — docs strongly discourage mixing.
  • Objects declared in YAML are reachable from Lua/console through getObjectFromYAMLConfiguration(name).
  • No "reload config file" primitive exists. Config is read at start; runtime changes are made through the console (Lua) and are lost on restart unless replayed (delta() prints them so they can be copied into the config).
• Top-level YAML keys (verified from the definitions file and docs): acl, backends, binds, cache_hit_response_rules, cache_inserted_response_rules, cache_miss_rules, cache_settings, console, dynamic_rules, dynamic_rules_settings, ebpf, edns_client_subnet, general, key_value_stores, load_balancing_policies, logging, metrics, mmdbs, netmask_groups, packet_caches, pools, proxy_protocol, query_count, query_rules, remote_logging, response_rules, ring_buffers, security_polling, selectors, self_answered_response_rules, snmp, timed_ip_sets, timeout_response_rules, tuning, webserver, xfr_response_rules, xsk.

Minimal kudurru-style dnsdist.yml (field names verified against the definitions file and dnsdist.yml-dist):

acl:
  - "0.0.0.0/0"
  - "::/0"
console:
  listen_address: "127.0.0.1:5199"
  key: "<base64 32 bytes>"          # makeKey() or openssl rand -base64 32
  acl: ["127.0.0.1", "::1"]
webserver:
  listen_addresses: ["0.0.0.0:8083"]
  api_key: "<hashed with hashPassword() or plaintext + hash_plaintext_credentials>"
  password: "<...>"
  acl: ["10.0.0.0/8"]
  stats_require_authentication: true
  api_read_write: false             # PUT allow-from only; backends are NOT API-managed
  prometheus_add_instance: true     # label from general.server_id
general:
  server_id: "dnsdist-node-a"
  capabilities_to_retain: []        # add CAP_SYS_ADMIN only if ebpf: is used
binds:
  - listen_address: "0.0.0.0:53"
    protocol: "Do53"
    reuseport: true
    threads: 4
  - listen_address: "0.0.0.0:853"
    protocol: "DoT"
    tls: { certificates: [{ certificate: "/certs/tls.crt", key: "/certs/tls.key" }] }
  - listen_address: "0.0.0.0:443"
    protocol: "DoH"
    tls: { certificates: [{ certificate: "/certs/tls.crt", key: "/certs/tls.key" }] }
    doh: { provider: "nghttp2", paths: ["/dns-query"] }
  - listen_address: "0.0.0.0:853"
    protocol: "DoQ"
    tls: { certificates: [{ certificate: "/certs/tls.crt", key: "/certs/tls.key" }] }
  - listen_address: "0.0.0.0:443"
    protocol: "DoH3"
    tls: { certificates: [{ certificate: "/certs/tls.crt", key: "/certs/tls.key" }] }
packet_caches:
  - name: "pc"
    size: 200000
    max_ttl: 86400
    min_ttl: 0
    temporary_failure_ttl: 60
    stale_ttl: 60
    shards: 20
pools:
  - name: ""                        # default pool
    packet_cache: "pc"
    policy: "leastOutstanding"
load_balancing_policies:
  default_policy: "leastOutstanding"
  servfail_on_no_server: true
backends:
  - address: "10.0.0.11:5353"
    name: "kudurru-0"
    id: "4f2c1d0e-0000-4000-8000-000000000000"   # stable UUID -> stable chashed ring
    protocol: "Do53"
    weight: 100
    order: 1
    use_proxy_protocol: true        # kudurru sees the real client IP
    sockets: 4
    health_checks:
      mode: "auto"                  # auto | up | down | lazy
      qname: "health.kudurru.internal."
      qtype: "A"
      interval: 1
      timeout: 1000
      max_failures: 2
      rise: 2
      must_resolve: true
query_rules:
  - name: "refuse xfr/notify/update from unknown"
    selector:
      type: "Or"
      selectors:
        - { type: "Opcode", code: "Notify" }
        - { type: "Opcode", code: "Update" }
        - { type: "QType", qtype: "AXFR" }
        - { type: "QType", qtype: "IXFR" }
    action: { type: "RCode", rcode: "Refused" }
metrics:
  carbon:
    - address: "graphite.internal:2003"
      name: "dnsdist-node-a"
      interval: 30
      namespace: "kudurru"
      instance: "dnsdist"

Note: the Or/Opcode/QType selector field names above follow the YAML selector catalogue (dnsdist-selectors-definitions.yml); I verified the QNameSuffix, QName, NetmaskGroup, Pool, Drop shapes from dnsdist.yml-dist. Validate every generated file with dnsdist --check-config -C file.yml in CI — 2.0.9/2.1.2 fixed naming bugs in exactly these And/Or/Not selectors when configured from YAML.

1.3 Backend (downstream) definition — full parameter list

Lua newServer({...}) keys and their YAML backends[] equivalents (verified in docs/reference/config.rst and the definitions file):

Runtime management (Lua/console): newServer(table) returns a Server; getServer(idx|uuid), getServers(), rmServer(idx|uuid|serverObj), showServers({showUUIDs=true}); Server:setUp()/setDown()/setAuto()/setActiveAuto()/setLazyAuto(), Server:addPool()/rmPool(), Server:setQPS(), Server:setHealthCheckParams({...}), Server:isUp(), Server:getOutstanding(), Server:getLatency(). The dnsdist UDP design keeps a per-backend IDState table, so at most 65,535 in-flight UDP queries per backend (docs/advanced/internal-design.rst).

1.4 Health checks

(docs/guides/downstreams.rst, reference/config.rst, definitions file — verified.)

• Modes (healthCheckMode / health_checks.mode): auto (active probe every checkInterval, default 1 s), up (always up), down (always down), lazy (added 1.8.0).
• Active probe: checkName (default a.root-servers.net.), checkType (A), checkClass (IN), checkTimeout 1000 ms, maxCheckFailures 1, rise 1, mustResolve false (default: any RCODE except ServFail is OK; with mustResolve NXDomain/Refused also fail), setCD, checkTCP (use_tcp; always TCP for DoT backends).
• checkFunction(qname, qtype, qclass, dh) Lua hook rewrites the probe (returns new qname/qtype/qclass, may mutate the header). YAML: health_checks.function|lua|lua_file.
• 2.1.0: Server:setHealthCheckResponseValidator(fn) / YAML validation_function|validation_lua|validation_lua_file — validate the response (e.g. require a specific A record or a TXT with the kudurru node's serial/epoch). Mutually exclusive options.
• 2.1.0: addServerStateChangeCallback(function(nameAddr, isUp) ... end) — a global hook; a Lua callback can POST to kudurru (via registerWebHandler's sibling, or just log) when a backend flips. Keep it non-blocking (it runs in the health-check thread).
• Lazy mode knobs: lazyHealthCheckSampleSize 100, lazyHealthCheckMinSampleCount 1, lazyHealthCheckThreshold 20 (% failures), lazyHealthCheckFailedInterval 30 s, lazyHealthCheckMode TimeoutOrServFail|TimeoutOnly, lazyHealthCheckUseExponentialBackOff, lazyHealthCheckMaxBackOff 3600. States: Healthy -> PotentialFailure -> Failed.
• 2.0.3 added a health-check latency metric; verboseHealthChecks is a Cloud Control knob mapping to setVerboseHealthChecks(true).
• Server-side JSON (/api/v1/servers/localhost) exposes healthCheckFailures, healthCheckFailuresParsing|Timeout|Network|Mismatch|Invalid per backend.

1.5 Pools

• Default pool is "". PoolAction("name") routes; QPSPoolAction(maxqps, "name") routes only under a QPS budget; per-pool policy setPoolServerPolicy(policy, pool); per-pool cache getPool("x"):setCache(pc); YAML pools[] {name, packet_cache, policy, use_ecs, use_zero_scope}.
• kudurru use: one pool per role or per zone-set, e.g. authoritative (kudurru primaries + secondaries), resolver (kudurru recursive/forwarding role), blocklist-bypass, abuse.

1.6 Load-balancing policies

(docs/guides/serverselection.rst — verified. Only servers marked up are candidates.)

There is no orderedWhashed policy in the docs (the task mentioned it — treat as a misnomer). setServFailWhenNoServer(true) (servfail_on_no_server) returns SERVFAIL instead of dropping.

kudurru recommendation: chashed with stable UUIDs for the resolver pool (cache locality on kudurru's own query cache), leastOutstanding for authoritative.

1.7 Rules, selectors, actions; eBPF / XDP / AF_XDP

• Pipeline (docs/rules-actions.rst): ACL -> query rules in order -> action (drop, answer, forward to pool, modify, delay). Rule chains: query_rules, response_rules, cache_hit_response_rules, cache_inserted_response_rules, self_answered_response_rules, xfr_response_rules, timeout_response_rules, cache_miss_rules.
• Selectors (reference/selectors.rst): QNameSuffixRule(list) (1.9.0; preferred over regex), QNameSetRule, SuffixMatchNodeRule, NetmaskGroupRule(nmg, src=true), QTypeRule, OpcodeRule, TCPRule, MaxQPSIPRule(qps, v4mask, v6mask), MaxQPSRule, RegexRule/RE2Rule, AndRule/OrRule/NotRule, AllRule, protocol selector (2.0.2), TimedIPSetRule, KeyValueStoreLookupRule, etc.
• Actions (reference/actions.rst): PoolAction(pool[,stop]), QPSPoolAction, RCodeAction(rcode), ERCodeAction, DropAction, DelayAction(ms) (UDP only), TCAction, SpoofAction(ips), SpoofRawAction, NegativeAndSOAAction, SetTagAction, SetECSAction, TeeAction(remote) (mirror traffic — handy for migrating onto kudurru), LuaAction, LuaFFIAction, ContinueAction, RemoteLogAction (protobuf), DnstapLogAction, SetProxyProtocolValuesAction, SetTraceAction (2.1).
• MaxQPSIPRule(5, 32, 48) + DelayAction(100) / SetNoRecurseAction() / TCAction() are the canonical per-client rate-limit idioms.
• Rule management: addAction(rule, action, {name=, uuid=}), showRules(), rmRule(id|uuid), mvRule, topRule; YAML rules carry name + uuid so a controller can address them.
• eBPF socket filter (docs/advanced/ebpf.rst): kernel-space drop by source IP (UDP+TCP) or exact qname (UDP only); needs CAP_SYS_ADMIN retained (addCapabilitiesToRetain / general.capabilities_to_retain), net.core.optmem_max, RLIMIT_MEMLOCK; not for DoQ/DoH3. newBPFFilter({ipv4MaxItems, ipv6MaxItems, qnamesMaxItems}), setDefaultBPFFilter(bpf); drop DynBlocks are pushed into it automatically since 1.6.0. Qtype 65535 = all types (changed in 2.0). Maps can be pinned (pinnedPath) so an external XDP program (contrib/xdp.py) answers TC=1 or drops before the stack.
• AF_XDP / XSK (docs/advanced/xsk.rst, 1.9.0+): newXsk({ifName, NIC_queue_id, frameNums, xskMapPath}) per NIC queue; needs CAP_NET_ADMIN+CAP_SYS_ADMIN+CAP_NET_RAW, symmetric routing, frames <= 2048 B. Advanced; not a default for kudurru.

1.8 Packet cache

(docs/guides/cache.rst + definitions — verified.) Per pool, shareable. newPacketCache(maxEntries, {maxTTL=86400, minTTL=0, temporaryFailureTTL=60, staleTTL=60, dontAge=false, shuffle=false, numberOfShards=20, deferrableInsertLock=true, maxNegativeTTL=3600, parseECS=false, keepStaleData=false, cookieHashing=false, maximumEntrySize=4096, optionsToSkip={10,12}}). YAML packet_caches[] uses snake_case (max_ttl, stale_ttl, shards, parse_ecs, payload_ranks...). Buckets pre-allocated (~dozen bytes/entry); setStaleCacheEntriesTTL(n) serves stale when no backend is up. Runtime: getPool("").getCache():expungeByName(newDNSName("x"), DNSQType.A), :purgeExpired(0), :expunge(0), :getStats(). HTTP: DELETE /api/v1/cache?pool=&name=[&type=&suffix=] (allowed even when API is read-only). kudurru must call this when a zone changes if the dnsdist cache is enabled in front of the authoritative pool — or set max_ttl low / disable the cache for the authoritative pool and keep it for the resolver pool.

1.9 Frontends: Do53, DoT, DoH, DoQ, DoH3, DNSCrypt

• Lua: addLocal(addr, {reusePort, tcpFastOpenQueueSize, interface, cpus, tcpListenQueueSize, maxInFlight, maxConcurrentTCPConnections, enableProxyProtocol, xskSocket}); addTLSLocal(addr, certs, keys, opts); addDOHLocal(addr, certs, keys, paths, opts) (default path /dns-query, nghttp2 only in 2.1+, internalPipeBufferSize=1048576 recommended under load, custom Alt-Svc/link headers); addDOQLocal / addDOH3Local (Quiche; congestionControlAlgo), addDNSCryptBind.
• YAML binds[]: listen_address, protocol (Do53|DoT|DoH|DoQ|DoH3|DNSCrypt), reuseport, threads, interface, cpus, enable_proxy_protocol, pad_responses, tcp:{...}, tls:{certificates:[{certificate,key}], provider, ciphers, ...}, doh:{provider, paths, custom_response_headers, ...}, doq, quic:{congestion_control_algorithm}, dnscrypt, additional_addresses, xsk.
• Certificates: reloadAllCertificates() from the console after cert-manager/ACME rotation; files must be readable by the pdns user (dnsdist drops root since 1.5.0). Session ticket keys (STEK) can be rotated/loaded (loadTicketsKey) — Cloud Control syncs a stekSecret into dnsdist.
• Forwarding protocol matrix (docs/running.rst): UDP->UDP, TCP/DoT/DoQ/DoH3->TCP, DoH->UDP with TCP fallback on TC; tcp_only/DoT/DoH backends override.

1.10 Dynamic blocking (DynBlocks)

(docs/guides/dynblocks.rst — verified.) Ring buffers (setRingBuffersSize(n, shards), YAML ring_buffers) are analysed once per second from maintenance():

local dbr = dynBlockRulesGroup()
dbr:setQueryRate(30, 10, "Exceeded query rate", 60)                 -- qps, window s, reason, block s
dbr:setRCodeRate(DNSRCode.NXDOMAIN, 20, 10, "Exceeded NXD rate", 60)
dbr:setRCodeRatio(DNSRCode.SERVFAIL, 0.5, 10, "SF ratio", 60, 100)   -- ratio with min responses
dbr:setQTypeRate(DNSQType.ANY, 5, 10, "Exceeded ANY rate", 60)
dbr:setResponseByteRate(10000, 10, "Exceeded resp BW rate", 60)
dbr:setSuffixMatchRule(10, "PRSD", 60, visitorFn)                   -- per-suffix (random-subdomain) detection
dbr:setAllowedRCodesRatio({DNSRCode.NOERROR}, 0.2, 10, "bad rcodes", 60, 100) -- 2.1.0
dbr:excludeRange({"10.0.0.0/8"}); dbr:includeRange({"10.1.2.3/32"})
dbr:setMasks(32, 64, 0)
function maintenance() dbr:apply() end

Actions: Drop (default; goes to eBPF when setDefaultBPFFilter is set), DNSAction.Refused, Truncate, NoRecurse, Nxdomain, ServFail, SetTag, warning-only rates. YAML dynamic_rules[] with type, rate/ratio, seconds, action_duration, action, warning_rate, rcode, qtype, allowed_rcodes, minimum_number_of_responses, minimum_global_cache_hit_ratio, visitor_function_*. Inspect/clear: showDynBlocks(), clearDynBlocks(), /jsonstat?command=dynblocklist. External trigger (1.9.0+): addDynamicBlock(addr, msg[, action[, seconds[, v4mask[, v6mask]]]]) behind a registerWebHandler('/block', fn) endpoint — this is how kudurru's UI/blocking engine can push a temporary client block to the edge.

1.11 Console (encrypted control socket)

(docs/guides/console.rst, reference/config.rst, manpage — verified.)

• Enable: Lua controlSocket('127.0.0.1:5199') + setKey("<base64>"); YAML console:{listen_address, key, acl, maximum_output_size 10000000, log_connections true, max_concurrent_connections 0, bind_fatal}. Default port 5199; default ACL 127.0.0.1, ::1; encryption needs libsodium or libcrypto build (the official image has -Dlibsodium=enabled). Key = any 32 random bytes base64.
• Client: dnsdist -c (reads the config to find socket+key), dnsdist -C cfg -c, dnsdist -k KEY -c host:5199 (leaks key into shell history — avoid), and non-interactive: dnsdist -C cfg -e 'newServer({address="10.0.0.12:5353", name="kudurru-2"})'.
• The console evaluates arbitrary Lua in the running process: newServer, rmServer, getServer(uuid):setDown(), addAction, rmRule, reloadAllCertificates(), getPool(""):getCache():expunge(0), showServers(), dumpStats(), grepq(), topQueries(), delta() (prints everything changed since startup).
• Security: it is root-equivalent for the dnsdist process; keep it on localhost inside the pod/ container (kudurru's sidecar controller talks to it over the pod-local loopback) or restrict with acl + the shared key over a private network.
• UNVERIFIED: whether every configuration directive (e.g. newServer) is accepted on the console when the process was started from a YAML file without enableLuaConfiguration(). The docs say the sibling .lua is loaded "without the ability to use configuration directives" and that enableLuaConfiguration() re-enables them; test this on 2.1.x before relying on console membership changes with YAML. The Lua-config path (dnsdist.conf) definitely supports it.

1.12 Webserver, REST API, Prometheus

(docs/guides/webserver.rst, reference/config.rst — verified.)

• Enable: Lua webserver("0.0.0.0:8083") + setWebserverConfig({password=, apiKey=, acl=, customHeaders=, apiRequiresAuthentication=true, dashboardRequiresAuthentication=true, statsRequireAuthentication=true, allowCrossOriginRequests=false, prometheusAddInstanceLabel=false, maxConcurrentConnections=100, hashPlaintextCredentials=false}); credentials should be hashPassword("...") (scrypt). YAML: webserver:{listen_addresses[], password, api_key, acl, api_requires_authentication, stats_require_authentication, dashboard_requires_authentication, prometheus_add_instance, max_concurrent_connections, hash_plaintext_credentials, custom_headers[], api_configuration_directory, api_read_write, bind_fatal}.
• No TLS on the built-in webserver — put it behind kudurru's own HTTPS (reverse proxy) or keep it on the pod network. Auth: HTTP basic (password only; username ignored) for dashboard, X-API-Key header for /api.
• Endpoints:
  • GET /jsonstat?command=stats|dynblocklist|ebpfblocklist
  • GET /metrics (Prometheus; dnsdist_* families incl. per-server dnsdist_server_*{server=,address=}, per-frontend dnsdist_frontend_*, per-pool cache metrics, dnsdist_info{version=}, latency buckets)
  • GET /api/v1/servers/localhost -> {acl, daemon_type:"dnsdist", version, frontends[], pools[], servers[], rules[], response-rules[], cache-hit-response-rules[], self-answered-response-rules[]}
    • servers[]: address, id (uuid), name, state ("up"|"down"), order, weight, qps, qpsLimit, queries, responses, outstanding, reuseds, sendErrors, dropRate, latency, tcpLatency, protocol, pools[], tcpCurrentConnections, tcpNewConnections, tcpAvgConnectionDuration, tlsResumptions, healthCheckFailures{,Parsing,Timeout,Network,Mismatch,Invalid}
    • pools[]: id, name, serversCount, cacheSize, cacheEntries, cacheHits, cacheMisses, cacheDeferredInserts, cacheDeferredLookups, cacheLookupCollisions, cacheInsertCollisions, cacheCleanupCount, cacheTTLTooShorts
    • frontends[]: address, id, type (UDP|TCP|DoT|DoH...), udp, tcp, queries, responses, nonCompliantQueries, tls10Queries..tls13Queries, tlsNewSessions, tlsResumptions, handshake failures
    • rules[]: id, name, uuid, rule, action, action-stats, matches, creationOrder
  • GET /api/v1/servers/localhost/statistics, .../config, .../config/allow-from, PUT .../config/allow-from (needs setAPIWritable(true[, dir]); persisted only if dir given), GET .../pool?pool-name=<name> -> {list:[metrics], servers:[...]}, GET .../rings?maxQueries=&maxResponses=
  • DELETE /api/v1/cache?pool=&name=&type=&suffix=
  • custom: registerWebHandler(path, luaFn) — arbitrary endpoints (used by the docs' dynamic block example)
• The REST API cannot create/delete backends, pools or rules. Membership changes = console or Lua.
• Carbon: carbonServer(ip[, ourname[, interval[, namespace[, instance]]]]), YAML metrics.carbon[]. SNMP agent optional (snmpAgent(true)). 2.1: OpenTelemetry traces (logging.open_telemetry_tracing.enabled).

1.13 Passing the client address to kudurru

(docs/advanced/passing-source-address.rst — verified.) Because dnsdist terminates UDP/TCP, kudurru would otherwise log every query as coming from dnsdist. Options: (1) PROXY protocol v2 per backend (useProxyProtocol=true / use_proxy_protocol: true) — a small binary header before each UDP datagram or at the start of a TCP connection, supported by BIND, Knot, Unbound, PowerDNS; kudurru must implement a PROXYv2 parser and restrict it to dnsdist's source IPs (PowerDNS uses proxy-protocol-from=). TLVs can carry extra values (SetAdditionalProxyProtocolValueAction, proxyProtocolAdvertiseTLS to tell kudurru the client used DoT/DoH). (2) ECS (useClientSubnet), weaker (source IP only, prefix-truncated by default /24,/56). Also setProxyProtocolACL lets dnsdist itself accept PROXY headers from an upstream L4 LB.

1.14 Docker images

(Dockerfile-dnsdist, Docker-README.md, dockerdata/ in the clone; Docker Hub pages — verified.)

• Images: powerdns/dnsdist-19 (1.9.x), powerdns/dnsdist-20 ("PowerDNS DNSdist v2.0.x", ~97.6 MB), powerdns/dnsdist-21 ("v2.1.x", ~100.7 MB); only the latest tag is advertised on Hub — pin by digest in kudurru manifests. Also powerdns/dnsdist-master.
• Build (master Dockerfile): Debian trixie-slim builder + runtime, clang/meson/Rust/Quiche, with -Ddns-over-https, -ddns-over-quic, -Ddns-over-http3, -Ddns-over-tls, -Ddnscrypt, -Ddnstap, -Dyaml, -Debpf, -Dxsk, -Dsnmp, -Dlibsodium, -Dotlp, -Dre2 enabled; LuaJIT on amd64, Lua 5.3 on arm64/s390x. (UNVERIFIED that the 1.9/2.0 release-branch images use the same base; the branch Dockerfiles may differ.)
• Runtime: tini entrypoint -> /usr/local/bin/dnsdist-startup (Python/Jinja2 startup.py) -> dnsdist --supervised --disable-syslog. Runs as user pdns (uid 953). EXPOSE 53/udp 53/tcp 5199/tcp 8083/tcp. Config: /etc/dnsdist/dnsdist.conf (Lua) = setLocal('0.0.0.0') + includeDirectory('/etc/dnsdist/conf.d'); mount your .conf files into conf.d/.
  • DNSDIST_API_KEY=<key> env -> writes conf.d/_api.conf with webserver("0.0.0.0:8083"), setWebserverConfig({password=key, apiKey=key, acl='0.0.0.0/0'}), controlSocket('0.0.0.0:5199'), setKey(key), setConsoleACL('0.0.0.0/0') — note this exposes console and API on all interfaces; override for production.
  • TEMPLATE_FILES=a,b renders /etc/dnsdist/templates.d/a.j2 with the environment into conf.d/a.conf.
  • DEBUG_CONFIG=yes prints the rendered config (secrets included).
  • The image ships /etc/dnsdist/dnsdist-resolver.lua: a Lua module that keeps resolver.servers[name]={pool=...} in sync with DNS (getAddressInfo) each maintenance() tick — adds/removes newServers by name (see 1.16).
• Capabilities: since 1.7.1 the image does not grant net_bind_service; Docker >= 20.10 lets non-root bind :53 anyway; in Kubernetes add NET_BIND_SERVICE or bind an unprivileged port and map via Service.
• Since YAML is the 2.1 default, mounting /etc/dnsdist/dnsdist.yml (and optionally dnsdist.lua for functions) and passing -C /etc/dnsdist/dnsdist.yml is the cleanest path for kudurru-generated config. UNVERIFIED: whether the image's default Lua dnsdist.conf is bypassed automatically when a dnsdist.yml is present in /etc/dnsdist; pass -C explicitly.

1.15 dnsdist in Kubernetes

What PowerDNS itself does (Cloud Control docs, doc.powerdns.com/cloudcontrol — summarised): dnsdist instance sets with replicas: 2, pod anti-affinity, HPA, PDB; each pod runs dnsdist, agent (logging + readiness probing), dnsdist-init, dnsdist-state, rpc-server containers; backends come from serverGroups referencing other instance sets with backendsPerServer; serverPurgeDelay (5 s) removes replaced backends after a delay and stopDelay drains connections; Services per listener (Do53/DoT/DoH/DoQ/DoH3) of type ClusterIP/LoadBalancer/NodePort, externalTrafficPolicy configurable, hostNetwork: false by default (optional). That is the same "sidecar agent reconciles backends into a running dnsdist" shape recommended below.

Patterns for kudurru:

• Front tier Deployment (recommended): dnsdist Deployment (2+ replicas, anti-affinity) with a kudurru dnsdist-agent sidecar; backends = kudurru StatefulSet pods via headless Service (kudurru-0.kudurru.ns.svc). Expose with a LoadBalancer Service (kube-vip/MetalLB) with both 53/UDP and 53/TCP ports in one Service (mixed-protocol LB Services are GA — UNVERIFIED exact version, believed v1.26; MetalLB alternatively supports two Services sharing an IP via metallb.io/allow-shared-ip). Use externalTrafficPolicy: Local so client IPs survive to dnsdist (and then PROXYv2 to kudurru).
• Per-node DaemonSet with hostNetwork (NodeLocal-DNSCache style): dnsdist binds the node IP / a link-local VIP on every node; no Service hop, no conntrack, best latency; costs: port 53 must be free on nodes, dnsPolicy: ClusterFirstWithHostNet for the pod itself, and the pod's own DNS must not point at itself.
• Sidecar in each kudurru pod: dnsdist and kudurru in the same pod share a network namespace, so only one can own :53; dnsdist takes :53 and forwards to kudurru on 127.0.0.1:5353. This gives every kudurru pod DoT/DoH/DoQ/DoH3 termination, packet cache and DynBlocks for free, but there is no cross-pod balancing — it is the right shape for the single-instance/edge profile, not for the HA front tier.
• UDP + kube-proxy caveat (UNVERIFIED, from operational experience): long-lived UDP conntrack entries can pin clients to a dead backend after pod churn; prefer externalTrafficPolicy: Local and short dnsdist health-check intervals, and let dnsdist (not kube-proxy) do the backend selection.

1.16 How a kudurru controller should (re)configure dnsdist on membership change

Options, ranked:

1. Console push (dnsdist -e) from the agent sidecar. The agent watches kudurru membership (etcd/k8s EndpointSlice) and diff-applies: newServer({address=..., name=..., id=<stable uuid>, pool=..., useProxyProtocol=true, healthCheckMode='auto', ...}) for joins, rmServer("<uuid>") for leaves, getServer("<uuid>"):setDown() for drain. Verify with GET /api/v1/servers/localhost (servers[].id/state). Exactly what Cloud Control's agent container does; cost: a console connection per change. Changes are lost on restart, so the agent also regenerates dnsdist.yml so a restart converges to the same state.
2. Lua maintenance loop inside dnsdist (zero-controller option): ship a Lua file that, every second, resolves a headless Service name (getAddressInfo('kudurru.ns.svc.cluster.local.', cb)) and reconciles newServer/rmServer — this is literally dockerdata/dnsdist-resolver.lua. Fits Docker-compose too (Docker's embedded DNS resolves service names to all replica IPs with docker compose up --scale). Limitation: DNS-driven, so the only per-backend attributes are the ones in the static table (pool/order/weight); UUIDs are not stable across IP changes.
3. Regenerate YAML + rolling restart for structural changes (new bind, TLS, cache size, pool/policy changes). dnsdist starts in well under a second; with 2+ replicas behind a VIP/LB a rolling restart is invisible. Validate with dnsdist --check-config before rollout.
4. registerWebHandler custom HTTP endpoint implemented in Lua (e.g. POST /kudurru/backends) that calls newServer internally — an HTTP-shaped variant of (1) with no key exchange other than the webserver API key; still requires Lua.

Health of the pair: use GET /api/v1/servers/localhost as the dnsdist readiness probe (requires X-API-Key), /metrics for Prometheus, and kudurru's health QNAME (health.kudurru.internal., answered by every kudurru node with its node-id in a TXT) as the dnsdist backend probe with must_resolve: true plus a 2.1 validation function that checks the node-id.

2. keepalived

2.1 Identity, versions, licensing

• Repo github.com/acassen/keepalived; license GPL-2.0+ (README badge "GPLv2+", COPYING).
• Versions (GitHub tags via API, verified): v2.4.3, v2.4.2, v2.4.1, v2.4.0, v2.3.4, v2.3.3. configure.ac on master: AC_INIT([Keepalived], [2.4.3], ...). Release-notes site dates: 2.4.0 22 Jun 2026, 2.4.1 26 Jun, 2.4.2 7 Jul, 2.4.3 20 Jul 2026; 2.3.4 10 Jun 2025. The keepalived.org download page still says 2.3.4 is "latest stable" (stale). Distros: Alpine 3.23 2.3.4, Arch testing 2.4.2.
• 2.4.0 highlights (keepalived.org release notes, verified): auth_hmac (HMAC-SHA256 trailer with time/monotonic anti-replay, multiple keys for rotation, file: and systemd credentials — the modern replacement for the removed-from-spec auth_pass), ${_ENV VAR} substitution in config, vrrp_delay_after_boot, large hardening pass (parser, netlink, notify script path validation, DBus methods denied by default), passwords masked in dumps.
• Architecture: parent supervisor + child processes for VRRP, LVS health checkers and BFD; epoll-driven. Three frameworks: VRRP v2/v3 (IPv4+IPv6, sync groups), LVS/IPVS L4 load balancing, BFD. kudurru only needs VRRP (no virtual_server blocks, no ip_vs module needed unless the image insists on it).

2.2 Configuration (keepalived.conf)

Verified in doc/man/man5/keepalived.conf.5.in (3,164 lines) and samples.

global_defs {
    router_id kudurru-node-a            # free-form identifier (used in SMTP/SNMP), not VRID
    enable_script_security              # refuse root scripts in non-root-writable paths
    script_user keepalived_script       # default user for vrrp_script/notify
    vrrp_garp_master_repeat 5           # GARP burst on MASTER (default 5)
    vrrp_garp_master_delay 5            # second burst after N s (default 5)
    vrrp_garp_master_refresh 60         # periodic GARP while MASTER (0=off) - helps stale ARP caches
    vrrp_version 3                      # v3 = sub-second advert intervals, IPv6 native
    vrrp_notify_fifo /run/keepalived/vrrp.fifo   # event stream: INSTANCE "VI_DNS" MASTER 150
    vrrp_notify_fifo_script /usr/local/bin/kudurru-vrrp-events   # keepalived runs it, passes fifo path
    vrrp_startup_delay 2                # optional; vrrp_delay_after_boot (2.4.0)
    max_auto_priority                   # realtime scheduling of the vrrp child (optional)
}

vrrp_script chk_dns {
    script "/usr/local/bin/kudurru-dns-probe --server 127.0.0.1:53 --qname health.kudurru.internal. --timeout 0.5"
    interval 1                          # seconds (ms resolution)
    timeout 1                           # seconds; default = interval
    weight -40                          # 0 = go to FAULT on failure; negative = subtract from priority
    fall 3
    rise 2
    user keepalived_script
    init_fail                           # start in failed state until first success
}

vrrp_track_process trk_dnsdist {
    process dnsdist                     # exact /proc/PID/comm match (not regex)
    weight 10                           # +10 while quorum met
    quorum 1
    delay 2                             # debounce fork/exec and restarts
    full_command                        # match /proc/PID/cmdline instead (optional)
}

track_file kudurru_role {               # kudurru writes an integer here: 0 ok, non-zero = demote
    file "/run/kudurru/vrrp-weight"
    weight -1                           # value * weight added to priority; <-253 => FAULT
    init_file 0
}

vrrp_instance VI_DNS {
    state BACKUP                        # let the protocol elect; priority decides
    interface eth0                      # may be omitted with unicast peers (non link-local)
    virtual_router_id 53                # 1-255, unique per L2 segment + address family
    priority 150                        # 1-254 (255 = address owner); differ by >=50 across nodes
    advert_int 1                        # seconds; VRRPv3 allows e.g. 0.5
    nopreempt                           # a recovered higher-priority node does NOT steal the VIP (state must be BACKUP)
    preempt_delay 30                    # alternative: delay preemption N s (0..1000)
    garp_master_refresh 30              # per-instance override of global garp settings
    unicast_src_ip 10.0.0.11            # unicast mode (no multicast on the segment required)
    unicast_peer {
        10.0.0.12
        10.0.0.13
    }
    unicast_fault_no_peer               # fault instead of silently falling back to multicast
    check_unicast_src                   # only accept adverts from listed peers
    auth_hmac {                         # 2.4.0+; mutually exclusive with authentication{}
        key 1 file:/etc/keepalived/keys/vrrp53
        active_key 1
        anti_replay time                # needs NTP; or 'monotonic'
        mode enforce                    # roll out receive-only -> permissive -> enforce
    }
    virtual_ipaddress {
        10.0.0.53/24 dev eth0 label eth0:dns
        2001:db8::53/64                 # must be in virtual_ipaddress_excluded if mixing families in v2
    }
    track_interface { eth0 }
    track_script   { chk_dns }
    track_process  { trk_dnsdist weight 10 }
    track_file     { kudurru_role }
    notify_master "/usr/local/bin/kudurru-vrrp notify" keepalived_script
    notify_backup "/usr/local/bin/kudurru-vrrp notify"
    notify_fault  "/usr/local/bin/kudurru-vrrp notify"
    notify_stop   "/usr/local/bin/kudurru-vrrp notify"
    notify        "/usr/local/bin/kudurru-vrrp notify"   # generic; args appended: GROUP|INSTANCE name MASTER|BACKUP|FAULT|STOP|DELETED priority
    notify_priority_changes true        # also emit MASTER_PRIORITY/BACKUP_PRIORITY on the FIFO
}

Key semantics (from the man page):

• Election: highest priority wins; MASTER down timer = 3 * advert_int + skew (configurable down_timer_adverts); with advert_int 1 failover ~3.5 s; VRRPv3 permits sub-second advert_int so sub-second failover is possible at the cost of more chatter.
• nopreempt only works when initial state is not MASTER. Use it for DNS: flapping VIPs are worse than an asymmetric steady state.
• Weights: vrrp_script weight 0 => FAULT on fall failures; positive weight added after rise successes; negative subtracted after fall failures; effective priority clamped to [1,254]. Instances in a sync group must use weight 0 trackers.
• track_file is the cheapest integration hook: the application writes a number to a file and keepalived re-reads it on modification (inotify). kudurru can lower its own VRRP priority when it is not healthy / not a voting member, or raise it when it is the etcd leader.
• vrrp_track_process watches process existence via the kernel proc-connector (no polling, no pgrep) — handy to tie the VIP to a running dnsdist or kudurru process when they are in the same PID namespace (in containers that means pid: host or a shared pod PID namespace; otherwise use vrrp_script).
• Notify: notify_* scripts get, appended after any configured args, GROUP|INSTANCE, name, MASTER|BACKUP|FAULT|STOP|DELETED, priority (always uppercase). The FIFO (vrrp_notify_fifo) delivers the same tuple as a line INSTANCE "VI_DNS" MASTER 150 in guaranteed order plus MASTER_RX_LOWER_PRI and priority-change events — prefer the FIFO + notify_fifo_script to per-event script spawning. 2.4.0 validates resolved symlink targets of notify scripts/FIFOs.
• include/includer/includea with wildcards: includer /etc/keepalived/conf.d/*.conf lets kudurru drop generated fragments without touching the main file (include_check levels guard against partial writes).
• Parameter substitution: $VAR/${VAR} definitions, ${_ENV VARNAME} (2.4.0) reads environment — useful in containers (KEEPALIVED_* env -> config without templating).
• Reload: SIGHUP (or kill -s $(keepalived --signum=RELOAD) $(cat /run/keepalived.pid)) re-reads config without dropping the VIP where unchanged; moving a VIP between instances needs two reloads. --config-test/-t validates. -X/--release-vips on stop; USR1 dumps state to keepalived.data, USR2 stats, --signum=JSON dumps JSON (--enable-json build). DBus (enable_dbus, org.keepalived.Vrrp1) exposes state/priority and can create/destroy instances (2.4.0 denies methods by default — configure explicitly).
• Interface-less/unicast: with unicast_peer the interface may be omitted (then VIPs must specify dev), useful on cloud VMs where multicast is blocked; use_vmac creates a macvlan with the RFC 5798 virtual MAC (00-00-5E-00-01-{VRID}) so switch MAC tables do not need to relearn on failover — but it is incompatible with many cloud/virtual switches (port security).

2.3 CLI (keepalived.8)

-f/--use-file FILE, -n/--dont-fork, -l/--log-console, -D/--log-detail, -G/--no-syslog, -d/--dump-conf, -t/--config-test[=FILE], -X/--release-vips, -s/--namespace NAME (run in a network namespace), -P/--vrrp (VRRP only — skip checker child), --signum=STOP|RELOAD|DATA|STATS|JSON. Container-friendly invocation (from upstream Dockerfile.in): keepalived --dont-fork --log-console -f /etc/keepalived/keepalived.conf (add -P for VRRP-only).

2.4 Running keepalived in Docker

Upstream (Dockerfile.in, docker/README — verified):

• make docker builds an Alpine image compiling keepalived from the release tarball with --enable-bfd, optional --enable-nftables, --enable-json, --enable-snmp, --enable-dbus; creates user keepalived_script; entrypoint keepalived --dont-fork --log-console -f /etc/keepalived/keepalived.conf.
• Run: docker run --name keepalived --cap-add=NET_ADMIN --net=host -d keepalived; mount -v /data/keepalived.conf:/etc/keepalived/keepalived.conf; may need --security-opt label=disable on SELinux hosts.
• Kernel modules (ip_tables, ip6_tables, xt_set, ip_vs) cannot be loaded from inside the container — only needed if you use iptables/ipset/IPVS features; plain VRRP does not need ip_vs.
• Shut keepalived down (SIGTERM) before removing the container, otherwise VIPs/iptables rules are left behind on the host.
• No official Docker Hub image is published by the project (UNVERIFIED — none referenced in the repo; third-party images below).

Third-party images (Docker Hub / GitHub, verified metadata via API where noted):

Recommendation: kudurru should publish its own minimal keepalived image (Alpine apk add keepalived or upstream make docker), with a kudurru-rendered config mounted read-only, rather than depending on env-var templating from a third party. Required runtime flags: network_mode: host, cap_add: [NET_ADMIN, NET_RAW, NET_BROADCAST] (the last for multicast; unicast mode may not need it — UNVERIFIED), pid: host only if using track_process, restart: unless-stopped, a stop_grace_period long enough for notify_stop/VIP release.

2.5 keepalived in Kubernetes vs kube-vip vs MetalLB

Facts (verified from kube-vip.io, metallb.io, GitHub API):

• kube-vip v1.2.4 (2026-09-16, Apache-2.0). Modes: ARP (L2; one elected leader holds the VIP, sends gratuitous ARP; leader election via client-go Lease — DaemonSet manifest sets vip_leaseduration 5, vip_renewdeadline 3, vip_retryperiod 1), BGP (all nodes advertise), routing-table/"wireguard" modes without election. Deployed as static pods (control plane) or a DaemonSet with hostNetwork: true and caps NET_ADMIN NET_RAW SYS_TIME; RBAC via ServiceAccount. Services mode (svc_enable) watches type: LoadBalancer; IPs come from kube-vip-cloud-provider (ConfigMap ranges) or kube-vip.io/loadbalancerIPs; svc_election=true (v0.5.0+) runs one election per Service so VIPs spread across nodes; with externalTrafficPolicy: Local only nodes hosting an endpoint take part — preserves client IP; loadBalancerClass: kube-vip.io/kube-vip-class; optional IPVS L4 round-robin for the control-plane VIP (lb_enable), DHCP VIPs (0.0.0.0 request), kube-vip.io/ignore. Failover: planned = VIP evacuated first; crash = next lease holder takes over (lease 5 s) + GARP; blog reports sub-second to few-second (UNVERIFIED precise numbers).
• MetalLB v0.16.0 (2026-05-20, Apache-2.0). L2 mode: one speaker per Service IP answers ARP/NDP; announcer chosen statelessly by sorting hashes of node+VIP over memberlist membership (split-brain possible if memberlist disagrees); failover "within a few seconds" (bug if > 10 s) driven by GARP and client neighbor-cache refresh; single-node ingress bandwidth bottleneck; externalTrafficPolicy: Local => only the leader's local pods receive traffic, source IP preserved. BGP mode gives true ECMP across nodes. Two Services can share an IP with metallb.io/allow-shared-ip (docs' own example is TCP/53 + UDP/53), or one mixed-protocol Service. CRDs: IPAddressPool, L2Advertisement (node selectors, interfaces), BGPPeer, BGPAdvertisement.
• keepalived as a DaemonSet (community charts, e.g. Rancher/Interlegis keepalived chart, Deckhouse's keepalived module): hostNetwork: true, caps NET_ADMIN NET_RAW NET_BROADCAST, nodeSelector on "VIP-capable" nodes, config in a ConfigMap with per-node priority derived from an env/downward-API value, VRRP adverts between nodes over the node network (unicast_peer list = node IPs, which the kudurru operator can render from the Node list). It is not Kubernetes- aware: no Lease, no Service integration, no awareness of pod placement; the VIP lands on a node and traffic then needs a hostPort/hostNetwork listener or a NodePort/ClusterIP hop.

Comparison for kudurru's "DNS VIP" need:

Verdict: in Kubernetes, kube-vip or MetalLB are better fits (they integrate with Service objects and endpoints, need no per-node config rendering, and are Apache-2.0). Still use keepalived in k8s only when (a) the cluster already runs VRRP VIPs for other things and operators want one mechanism, (b) the VIP must fail over to a non-Kubernetes kudurru node (mixed estate: a docker-compose node as the last-resort backup of a k8s cluster — VRRP is the only protocol both speak), (c) the node network blocks ARP-spoofing from pods but allows VRRP, or (d) you need application-driven priority (track_file) rather than readiness-driven election. The kudurru Helm chart should therefore offer vip.provider: none|kube-vip|metallb|keepalived, defaulting to "none" (assume an existing LoadBalancer implementation) and generating keepalived config only on request.

2.6 Templating keepalived.conf from kudurru cluster state

Inputs kudurru already has (from its etcd/membership): node list with IPs and roles, the VIP(s), the local node identity, interface (configurable; default = interface owning the node IP), a shared secret. Render per node:

global_defs {
    router_id {{ .Node.Name }}
    enable_script_security
    script_user keepalived_script
    vrrp_version 3
    vrrp_garp_master_refresh 30
    vrrp_notify_fifo {{ .RunDir }}/vrrp.fifo
    vrrp_notify_fifo_script "{{ .Bin }} vrrp fifo"
}
{{- range .VIPs }}
vrrp_script chk_{{ .Name }} {
    script "{{ $.Bin }} health --dns 127.0.0.1:{{ .ProbePort }} --qname {{ .ProbeName }}"
    interval 1
    timeout 1
    fall 3
    rise 2
    weight 0
}
track_file tf_{{ .Name }} {
    file "{{ $.RunDir }}/weight-{{ .Name }}"
    weight -1
    init_file 0
}
vrrp_instance {{ .Name }} {
    state BACKUP
    interface {{ $.Interface }}
    virtual_router_id {{ .VRID }}
    priority {{ index .Priorities $.Node.Name }}
    advert_int {{ .AdvertInt }}
    {{- if .NoPreempt }}
    nopreempt
    {{- end }}
    unicast_src_ip {{ $.Node.IP }}
    unicast_peer {
    {{- range $.Peers }}
        {{ .IP }}
    {{- end }}
    }
    unicast_fault_no_peer
    check_unicast_src
    auth_hmac {
        key 1 file:{{ $.SecretDir }}/vrrp-{{ .VRID }}
        active_key 1
        anti_replay monotonic
        mode {{ .HMACMode }}
    }
    virtual_ipaddress {
        {{ .Address }} dev {{ $.Interface }}
    }
    track_script { chk_{{ .Name }} }
    track_file   { tf_{{ .Name }} }
}
{{- end }}

Rules for the generator:

• Priorities: deterministic from a stable ordering (e.g. member index) — 200 - 10*idx; differ by at least 10; never 255 (address owner semantics disable tracking adjustments).
• VRID: allocate per VIP from the cluster config (1-255), unique per L2 segment; store it in etcd so it never changes on re-render.
• unicast_peer = all other kudurru nodes on that segment; re-render and SIGHUP keepalived on membership change (the agent sidecar does kill -HUP <pid> via pid: host/shared PID namespace or sends docker kill -s HUP; in k8s the keepalived container shares the pod PID namespace with the agent (shareProcessNamespace: true)).
• Validate with keepalived -t -f <file> before swapping files; write atomically (rename) and keep the previous file for rollback.
• Use track_file as the kudurru -> keepalived channel: kudurru writes 0 when it is a healthy serving member, 60 (x weight -1 => -60) when draining, 300 (=> FAULT, since < -253) when it must not hold the VIP (e.g. not in quorum, maintenance mode).
• Consume state via the notify FIFO: on MASTER kudurru can (a) log/emit a metric "vip_owner", (b) trigger reloadAllCertificates/nothing on dnsdist, (c) update an integration (e.g. push the VIP's owner into Cloudflare/UniFi records). Do not block in notify scripts.

2.7 DNS health-check scripts for keepalived

• Shell with dig/kdig (bind-tools/knot-utils in Alpine): dig +time=1 +tries=1 +short @127.0.0.1 -p 53 health.kudurru.internal. A | grep -q '^127\.' || exit 1. Keep timeout >= tool timeout; keepalived kills scripts exceeding timeout and counts a failure.
• Prefer a built-in kudurru subcommand (kudurru health --dns 127.0.0.1:53 --qname ... --expect-txt node-id) — single static binary, no shell quoting issues, can also check dnsdist's /api/v1/servers/localhost to require "at least one backend up" so the VIP moves away from a node whose dnsdist has no healthy backends (exit 1 otherwise). Mount the kudurru binary read-only into the keepalived container (or build the keepalived image FROM a stage that copies it — the binary stays a separate program, so the GPL boundary holds).
• With enable_script_security, the script path must not be writable by non-root users and runs as keepalived_script; make sure that user can read the API key file.
• Probe what clients use: the VIP's :53 UDP and TCP (two checks or one tool that does both), not only loopback, since a misbound dnsdist would pass a loopback-only check.

3. Recommended topologies

3.1 Single instance — docker run / compose (edge profile)

client --Do53/DoT/DoH/DoQ--> [dnsdist :53/:853/:443] --PROXYv2 UDP/TCP--> [kudurru :5353]
                                   |  console 127.0.0.1:5199 / API 127.0.0.1:8083
                                   +<-- kudurru renders /etc/dnsdist/dnsdist.yml at start

• Simplest: one container? No — GPL boundary and upstream images argue for two containers in one compose project (or a pod). dnsdist is optional in the single-instance profile: kudurru can serve Do53/DoT/DoH itself; the "with-dnsdist" variant adds DoQ/DoH3, packet cache, DynBlocks and eBPF drops without kudurru implementing them.
• compose sketch:

services:
  kudurru:
    image: ghcr.io/kudurru/kudurru:1.0
    command: ["kudurru", "serve", "--listen", "0.0.0.0:5353", "--proxy-protocol-from", "127.0.0.1/32,::1/128"]
    volumes: ["kudurru-data:/var/lib/kudurru", "dnsdist-conf:/etc/dnsdist"]   # kudurru writes dnsdist.yml here
    environment: { KUDURRU_DNSDIST_CONSOLE: "127.0.0.1:5199", KUDURRU_DNSDIST_API: "http://127.0.0.1:8083" }
    network_mode: "service:dnsdist"          # share netns so 127.0.0.1 works both ways
  dnsdist:
    image: powerdns/dnsdist-21@sha256:<digest>
    command: ["-C", "/etc/dnsdist/dnsdist.yml"]
    ports: ["53:53/udp", "53:53/tcp", "853:853/tcp", "853:853/udp", "443:443/tcp", "443:443/udp"]
    volumes: ["dnsdist-conf:/etc/dnsdist:ro", "certs:/certs:ro"]
    depends_on: [kudurru]

(network_mode: service: sharing is the compose analogue of a pod; alternatively put both on a user-defined network and use service names.) kudurru generates the console key and API key once and stores them in its data dir.

3.2 docker-compose HA — 2-3 nodes + dnsdist + keepalived VIP

                 VIP 10.0.0.53 (VRRP VRID 53, unicast peers)
   node-a (prio 200)            node-b (prio 190)            node-c (prio 180)
   +-----------------+          +-----------------+          +-----------------+
   | keepalived (host net, NET_ADMIN)  track_script -> kudurru health          |
   | dnsdist :53/:853/:443  --> pool "kudurru": a,b,c (chashed, PROXYv2)       |
   | kudurru :5353 + embedded etcd/raft member                                  |
   +-----------------+          +-----------------+          +-----------------+

• Every node runs the same three containers; each dnsdist lists all kudurru nodes as backends (not just its local one), so a node whose kudurru is unhealthy but whose dnsdist is fine still serves; keepalived only decides who owns the VIP (and optionally a second VIP with reversed priorities for active/active across two addresses — publish both as NS/resolver addresses).
• keepalived container: network_mode: host, cap_add: [NET_ADMIN, NET_RAW, NET_BROADCAST], volumes: [./keepalived.conf:/etc/keepalived/keepalived.conf:ro, kudurru-bin:/opt/kudurru:ro], pid: host only if track_process is used. kudurru's agent renders keepalived.conf and dnsdist.yml from membership and sends HUP/console updates on change.
• dnsdist container: host networking is recommended on the HA nodes so it can bind the VIP (binds[].listen_address: 0.0.0.0:53 covers the VIP as it appears; or bind the VIP explicitly with ip_nonlocal_bind=1 sysctl so dnsdist can bind it before it is MASTER). Published-port mode also works since Docker's DNAT applies to the VIP, but UDP conntrack + VIP moves are messier.
• Backend health: health_checks.mode: auto, interval: 1, max_failures: 2, rise: 2, must_resolve: true, qname health.kudurru.internal.; keepalived vrrp_script probes the local dnsdist on the VIP/port 53 and the dnsdist API for "any backend up".
• Membership change flow: kudurru node joins -> raft membership updated -> every node's agent: (1) dnsdist -e 'newServer({...})' on its local dnsdist (idempotent by UUID), (2) rewrite dnsdist.yml for restart convergence, (3) rewrite keepalived.conf unicast_peer list and priorities, keepalived -t, then HUP.

3.3 Kubernetes — kudurru StatefulSet + dnsdist Deployment + VIP

   LoadBalancer Service dns-vip (53/UDP+TCP, 853, 443; externalTrafficPolicy: Local)
        provided by kube-vip (svc_election) | MetalLB (L2/BGP) | cloud LB | keepalived DaemonSet (opt-in)
                 |
   Deployment dnsdist (2-3 replicas, anti-affinity, PDB)   <-- Pod: dnsdist + kudurru-dnsdist-agent sidecar
                 |  PROXYv2 to kudurru-{0..n}.kudurru-headless:5353 (backends from EndpointSlice watch)
   StatefulSet kudurru (3 replicas, headless Service, PVCs, embedded etcd or k8s Lease leader election)
   CRDs: Zone, Record, Blocklist, ClientGroup, Integration; operator = kudurru itself (leader-elected)

• dnsdist pod: powerdns/dnsdist-21 container (-C /etc/dnsdist/dnsdist.yml from an emptyDir written by the agent's init step; TLS from a cert-manager Secret; securityContext.capabilities.add: [NET_BIND_SERVICE] or bind 5353 and let the Service map 53->5353; readinessProbe = HTTP GET /api/v1/servers/localhost with X-API-Key (httpHeaders) or a dig exec probe), plus the agent (kudurru binary, kudurru dnsdist-agent) which: watches the kudurru headless EndpointSlice and the KudurruCluster CR, renders YAML, pushes newServer/rmServer over the pod-local console, calls DELETE /api/v1/cache on zone changes it hears about from kudurru, runs reloadAllCertificates() when the mounted cert changes, and exposes nothing externally. shareProcessNamespace: true is not needed (console over loopback).
• Prometheus: PodMonitor on dnsdist :8083/metrics (basic auth secret) and on kudurru.
• VIP choice (see 2.5): default to the cluster's LoadBalancer implementation; chart values vip.provider=kube-vip adds kube-vip.io/loadbalancerIPs annotation; vip.provider=metallb adds metallb.io/loadBalancerIPs (+ allow-shared-ip when splitting TCP/UDP Services); vip.provider=keepalived deploys a DaemonSet (hostNetwork, NET_ADMIN/NET_RAW/NET_BROADCAST, nodeSelector kudurru.io/vip=true, ConfigMap rendered by the operator with unicast_peer = selected node IPs and track_script = kudurru health --dns <nodeIP>:53) and switches the dnsdist Deployment to a DaemonSet with hostNetwork: true on the same nodes so the VIP owner actually listens on :53 (with ClusterIP-only dnsdist, a keepalived VIP would have nothing bound).
• Single-node k3s/edge profile: same manifests with 1 replica each; dnsdist as a sidecar of the kudurru pod is acceptable here (section 1.15).

4. Licensing boundary

• dnsdist: GPL-2.0 (pdns COPYING); keepalived: GPL-2.0-or-later (COPYING, README badge). kudurru: Apache-2.0. GPL-2.0 and Apache-2.0 are incompatible for linking/combining into one program; running them as separate processes that communicate over sockets, files and CLI arguments is the classic "mere aggregation" case. Therefore:
  • Never embed dnsdist/keepalived code or headers in the kudurru binary; no cgo bindings; no Lua libraries copied from the pdns tree (the dnsdist-resolver.lua pattern must be re-implemented, not copied).
  • Invoke them via the console protocol (dnsdist -e, or a from-scratch client implementing the documented console framing — UNVERIFIED whether a third-party client exists; simplest is to exec the dnsdist binary inside the dnsdist container), the HTTP API, signals (HUP), files (track_file, FIFO) and the notify script interface.
  • Shipping upstream images unmodified (pinned by digest) is cleanest. If kudurru publishes its own keepalived image (Alpine package), it must ship the GPL text and offer source (Alpine's apk source packages satisfy this) and document the two licences separately in the image README.
  • Helm charts / compose files referencing GPL images carry no licence obligation on kudurru.
• kube-vip and MetalLB are Apache-2.0: fine to recommend, bundle manifests, or even import Go packages (MetalLB's speaker is not a library; kube-vip has Go packages but is a daemon — do not link either; use them as-is).

5. What kudurru should borrow

1. dnsdist's YAML as the generated edge config (backends, binds, pools, packet_caches, query_rules with name+uuid, dynamic_rules, webserver, console), rendered from kudurru cluster state, validated with --check-config in CI and at runtime before rollout.
2. Stable backend UUIDs + chashed for the resolver pool (cache locality, minimal remap on join/leave); leastOutstanding for authoritative; order for primary-preferred secondaries.
3. PROXY protocol v2 from dnsdist to kudurru (and from any L4 LB to dnsdist): implement a PROXYv2 parser in kudurru's listener with an allow-list; this is what makes Pi-hole-style per-client query logs correct behind a load balancer.
4. Health-check design: dedicated health.kudurru.internal. zone answered locally by every node with a TXT containing node-id/serial; must_resolve: true, 2.1 validation_function to check the TXT; lazy mode for large fleets; addServerStateChangeCallback to surface flaps.
5. Console-driven membership changes + YAML regeneration (Cloud Control's agent pattern, serverPurgeDelay-style delayed removal to let in-flight queries finish; setDown() to drain before rmServer).
6. DynBlocks as the edge rate limiter (query/NXDOMAIN/SERVFAIL/ANY/byte rates, PRSD suffix detection, 2.1 allowed-rcode ratio), with eBPF offload on Linux, and addDynamicBlock via a registerWebHandler endpoint so kudurru's UI can "block this client for 10 min at the edge".
7. Packet cache invalidation hook: DELETE /api/v1/cache?pool=&name=&suffix= on zone change; short max_ttl for authoritative pools.
8. Prometheus /metrics with prometheus_add_instance + server_id, scraped alongside kudurru's own metrics; Carbon optional. Reuse dnsdist's metric naming where kudurru exposes comparable counters (*_queries, *_responses, latency buckets) so one Grafana board works.
9. TeeAction for migration: mirror production traffic from an existing resolver into kudurru.
10. From keepalived: the track_file + notify FIFO contract (application writes an integer, daemon emits ordered state lines) — adopt the same contract shape for kudurru's generic "VIP provider" interface so kube-vip/MetalLB/keepalived are interchangeable behind one abstraction (vip.Owner(), vip.SetWeight(), events).
11. From keepalived 2.4: auth_hmac with file: keys (secrets out of config) and ${_ENV}; kudurru should generate per-VRID HMAC keys and distribute them through its own secret store.
12. From kube-vip: per-Service election (svc_election) + externalTrafficPolicy: Local — the model for "one VIP per role" (authoritative VIP, resolver VIP) spread across nodes.
13. The upstream Docker image's startup contract (env -> rendered fragment in conf.d/, DEBUG_CONFIG, tini, non-root uid) as the template for kudurru's own image.

6. What kudurru should avoid

1. Do not link or vendor dnsdist/keepalived code (GPL-2.0 vs Apache-2.0); do not copy dnsdist-resolver.lua verbatim; always separate processes/containers.
2. Do not treat the dnsdist REST API as a configuration API — it cannot add/remove backends, pools or rules; only allow-from and cache flush are writable. Plan around the console/Lua.
3. Do not rely on an in-place YAML reload — there is none. Design for "console diff + regenerate file + occasional rolling restart".
4. Do not expose the console (5199) or webserver (8083) beyond loopback/pod network (the stock image's DNSDIST_API_KEY convenience sets 0.0.0.0/0 ACLs for both — override). The console is arbitrary-Lua-execution; the webserver has no TLS.
5. Do not mix Lua configuration directives with YAML (upstream says so; subtle breakage) — Lua companion = functions only (maintenance, health-check functions, web handlers).
6. Do not assume HTTP/1.1 DoH in 2.1+ (nghttp2 only); if kudurru terminates HTTPS in front of dnsdist it must speak h2 to it, or let dnsdist terminate TLS directly.
7. Do not enable eBPF/XSK by default: needs CAP_SYS_ADMIN/CAP_NET_ADMIN retained, sysctls, pinned BPF maps; make it an opt-in "performance" profile.
8. Do not cache authoritative answers in dnsdist without an invalidation path (stale data after zone edits); and never cache AXFR/IXFR (dnsdist already refuses UDP XFR in 2.2).
9. Do not use keepalived auth_pass (8-char, removed from the VRRP spec; only "helpful" for unicast) — use 2.4 auth_hmac, or at least check_unicast_src + unicast_fault_no_peer.
10. Do not set state MASTER on every node or use equal priorities; use state BACKUP, distinct priorities and nopreempt for DNS VIPs (avoid VIP flapping during node recovery).
11. Do not run keepalived on bridge networks (VRRP needs the host namespace); do not forget to stop keepalived before removing its container (VIP left behind); do not depend on docker.sock-driven notify scripts as some third-party images demo.
12. Do not make keepalived the default VIP mechanism in Kubernetes — kube-vip/MetalLB are Service-aware and Apache-2.0; keep keepalived opt-in for mixed estates.
13. Do not use priority 255 (address owner: tracking cannot adjust it) and do not reuse a VRID on the same segment/address family.
14. Do not run a dnsdist sidecar and expect cross-pod balancing; the sidecar shape is for the single-instance profile only.
15. Do not depend on latest tags (powerdns/dnsdist-21, osixia/keepalived) — pin digests; track dnsdist security releases (several CVE batches in 2026: 1.9.12/2.0.3, 1.9.13/2.0.4, 1.9.15/2.0.7).

7. Open questions / UNVERIFIED

• Whether console newServer/rmServer work at runtime when dnsdist was started from YAML without enableLuaConfiguration() (2.1.x). Needs a quick test; affects option (1) in 1.16.
• Whether the upstream Docker image auto-selects /etc/dnsdist/dnsdist.yml over the shipped Lua dnsdist.conf (pass -C explicitly regardless).
• Base image of the release-branch dnsdist-19/20/21 images (master Dockerfile uses Debian trixie-slim).
• A documented wire protocol for the console (to write a native Go client instead of exec'ing dnsdist -e); framing is libsodium/libcrypto secretbox-style per docs, but no spec was read.
• Mixed-protocol (TCP+UDP) LoadBalancer Service GA version (believed 1.26) and each VIP provider's behaviour with it; MetalLB documents the shared-IP workaround explicitly.
• Exact kube-vip ARP failover timing and MetalLB L2 timing under load (vendor blogs only).
• Whether NET_BROADCAST is required for keepalived in unicast-only mode (third-party images add it for multicast; upstream README only mentions NET_ADMIN).
• No official keepalived Docker Hub image was found; the project provides make docker only.
• dnsdist EOL policy per train (docs/eol.rst not read) — matters for choosing 2.0 vs 2.1 images.

8. Sources

• Clone: /tmp/kudurru-research/dnsdist-keepalived/pdns (HEAD 3fe6ea92, 2026-10-09): pdns/dnsdistdist/docs/{changelog,upgrade_guide,running,rules-actions,statistics}.rst, docs/guides/{downstreams,serverselection,serverpools,cache,console,webserver,carbon,dynblocks,dns-over-https,dns-over-tls,dns-over-quic,dns-over-http3}.rst, docs/advanced/{ebpf,xsk,tuning,internal-design,multiple-instance,passing-source-address}.rst, docs/reference/{config,actions,selectors,rules-management}.rst, docs/manpages/dnsdist.1.rst, pdns/dnsdistdist/dnsdist-settings-definitions.yml, dnsdist.yml-dist, dnsdist.conf-dist, Dockerfile-dnsdist, Docker-README.md, dockerdata/{dnsdist.conf,startup.py,dnsdist-resolver.lua}, COPYING.
• Clone: /tmp/kudurru-research/dnsdist-keepalived/keepalived (HEAD 2d0b4792, 2026-09-24): README.md, configure.ac, Dockerfile.in, docker/{README,keepalived.conf}, doc/man/man5/keepalived.conf.5.in, doc/man/man8/keepalived.8.in, doc/samples/{keepalived.conf.vrrp.scripts,keepalived.conf.vrrp.localcheck,sample_notify_fifo.sh}, COPYING.
• https://dnsdist.org/changelog.html, https://dnsdist.org/reference/yaml-settings.html, https://dnsdist.org/guides/downstreams.html, https://dnsdist.org/guides/serverselection.html, https://dnsdist.org/guides/webserver.html, https://dnsdist.org/guides/console.html, https://dnsdist.org/guides/dynblocks.html
• https://blog.powerdns.com/2026/07/02/powerdns-dnsdist-2.1.0-released, https://mailman.powerdns.com/pipermail/dnsdist/2026-March/001657.html (1.9.12/2.0.3 security), https://mailman.powerdns.com/pipermail/dnsdist/2026-April/001659.html (1.9.14/2.0.5)
• https://hub.docker.com/r/powerdns/dnsdist-20, https://hub.docker.com/r/powerdns/dnsdist-21
• https://doc.powerdns.com/cloudcontrol/latest/userplane/loadbalancing/dnsdist/
• https://keepalived.org/release-notes/, https://www.keepalived.org/release-notes/Release-2.4.0/, GitHub API tags for acassen/keepalived
• https://hub.docker.com/r/osixia/keepalived (+ GitHub API: MIT, pushed 2026-10-07), https://github.com/shawly/docker-keepalived (GPL-3.0, pushed 2026-07-31)
• https://kube-vip.io/docs/about/architecture/, https://kube-vip.io/docs/installation/daemonset/, https://kube-vip.io/docs/usage/kubernetes-services/; GitHub releases kube-vip v1.2.4 (2026-09-16)
• https://metallb.io/concepts/layer2/, https://metallb.io/usage/; GitHub releases MetalLB v0.16.0 (2026-05-20)
• https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/