Observability

Three signals, three homes, and none of them goes through the store:

Plus the audit trail of who changed what. Code: internal/stages/log, internal/querylog (collector, privacy, tail) and internal/querylog/sqlitelog (writer, rollups, retention, reader), internal/metrics, internal/health, internal/audit.

The query log pipeline

flowchart LR
    OBS["log observer<br/>(after every query)"] -->|"non-blocking send"| CH[["bounded channel<br/>10 000 events"]]
    CH -->|"full: drop and count"| DROP["kudurru_querylog_dropped_total<br/>{stage=queue}"]
    CH --> COL["collector (one goroutine)<br/>exclusions, privacy mode,<br/>stamp the ID"]
    COL --> TAIL["live tail<br/>ring of 65 536 + SSE subscribers"]
    COL --> BATCH["batch: 1000 events or 1 s"]
    BATCH --> DB[("querylog.db<br/>dictionaries + query table<br/>+ rollups, one transaction")]
    TAIL --> API["dashboard: live tail,<br/>block page reasons"]
    DB --> API2["dashboard: query log,<br/>stats, top lists"]

The answer never waits for the log: when the channel is full, the event is dropped and counted. The health name and the dashboard's built-in dig are never logged.

Privacy is applied once, at the collector, before anything is stored or shown: full, anonymize_client (a /24 or /48 prefix, or a daily pseudonym), hide_domains, hide_domains_clients, or nothing (only the per-minute totals). Changing the mode does not rewrite queries already logged.

What one event holds

Time, latency, node, transport, client address, client group, policy, name, type, class, rcode, status, upstream (<group>/<server>), zone, matched list and rule, answer count and a short answer summary, the first EDE code, flags (RD, CD, DO, ECS, TC, AD, cached, stale, prefetch) and the ECS prefix. The status, list and rule are what let the dashboard explain an answer in one line.

Storage and retention

Names, clients, upstreams, zones, lists and policies are stored once in dict_* tables and referenced by ID. Charts read the rollups, so a 30-day chart does not scan 30 days of rows. The database is per node: in a multi-node install each node logs what it answered (scatter-gather across nodes is planned).

Metrics

All metric labels are bounded: list names, upstream names and status values, never client addresses or query names.

deploy/prometheus/rules.yaml and deploy/grafana/kudurru-overview.json (also in the chart) are generated and tested against these names.

Health

stateDiagram-v2
    direction LR
    [*] --> Starting
    Starting --> Ready: zonecache initialised,<br/>every readiness check passes
    Ready --> NotReady: a check fails<br/>(objects, zonecache, blocklists,<br/>blocking, dns listeners)
    NotReady --> Ready: it recovers
    Ready --> Lameduck: kudurru lameduck / SIGTERM
    Lameduck --> [*]: drain time passes
    note right of Lameduck
        /healthz/ready 503 and the
        health name answers SERVFAIL,
        so dnsdist and Kubernetes
        both stop sending traffic
    end note

Upstream reachability is never a readiness condition: a node whose upstreams are down still answers its zones and its cache, and taking it out of rotation would make things worse.

Audit

Store writes are audited by the driver. The API adds what no store write covers: denied requests, the blocking toggle, cache flushes, query-log purges, NOTIFY and list refreshes.