Configuration and the store

kudurru separates three kinds of state, and keeps them apart on purpose:

This page covers the first two: how an object gets from a writer to every node's query path, and how settings reload.

Code: api/v1alpha1, api/internalkinds/v1alpha1, internal/store (interface, drivers, validator, conformance suite), internal/zonecache, internal/objects (the file loader), internal/bootstrap, internal/controller, internal/config, cmd/kudurru/reload.go.

The objects

All kinds share kudurru.io/v1alpha1. Public kinds are CRDs in Kubernetes; internal kinds hold credentials and are never CRDs (Secrets in Kubernetes, rows in state.db in Docker).

erDiagram
    DNSZone ||--o{ DNSRecordSet : "zoneRef"
    Policy }o--o{ ClientGroup : "clientGroupRefs"
    Policy }o--o{ BlockList : "blockListRefs / allowListRefs"
    UnblockRequest }o--o| BlockList : "status.allow.list"
    UnblockRequest }o--o| ClientGroup : "status.allow.group"
    APIToken }o--o| LocalUser : "owner (User)"
    APIToken }o--o| ServiceAccount : "owner (ServiceAccount)"
    RoleBinding }o--o{ ServiceAccount : "subjects"
    RoleBinding }o--o| DNSZone : "scope zone:<name>"

The bootstrap seeds four objects on the first start, with owner bootstrap: the root forward zone . → default, ClientGroup default (all addresses), BlockList starter and Policy default.

One writer per object

Every write carries a manager, recorded as the object's owner in the kudurru.io/managed-by annotation: api, file:<path>, integration:<name>, bootstrap or system.

flowchart LR
    W(["Apply with manager M"]) --> O{"recorded owner"}
    O -- "empty (first write)" --> OK["write, owner = M"]
    O -- "equal to M" --> OK
    O -- "someone else" --> F{"Force?<br/>(admin, audited)"}
    F -- "yes" --> OK2["write, owner = M"]
    F -- "no" --> E["409 managed_by<br/>hint names the owner"]

This is why a zone from objects.d/zones.yaml cannot be edited in the dashboard ("managed by file objects.d/zones.yaml"), and why two sources of truth for one object cannot exist. The API may take over objects owned by bootstrap. Objects written with kubectl carry no annotation, so anyone may write them.

The store

// internal/store/store.go (abridged)
type Store interface {
    Get, List, Apply, Delete, UpdateStatus  // CRUD; Apply is whole-object, optimistic (resourceVersion)
    Watch(ctx, kind, opts) (Watcher, error) // list + watch: ADDED, MODIFIED, DELETED, BOOKMARK
    Elector(name string) Elector             // leader election primitive
    Snapshot, Restore                        // backup = `kind: List` YAML that `kudurru apply -f` accepts
    Info() Info; Close() error
}

Two drivers exist. They pass the same conformance suite (internal/store/conformance), so the rest of the code cannot tell them apart.

The validator (internal/store/validator) runs the apiserver's own structural schema, defaulting and CEL programs from the embedded CRDs, so Docker rejects exactly what Kubernetes rejects. The API, the file loader, the bootstrap and the sqlite driver all call it. Cross-object problems (a record outside its zone, a duplicate) are not validation errors: the controller reports them as conditions.

Each database carries a schema version. A file newer than the binary stops the process with an explanation; an older one is migrated in one transaction.

From a write to the query path

sequenceDiagram
    autonumber
    participant U as dashboard / CLI
    participant API as API :8080
    participant V as validator
    participant S as store
    participant ZC as zonecache (every node)
    participant P as pipeline
    U->>API: POST /api/v1/apply (YAML)
    API->>V: strict decode, default, schema + CEL
    API->>S: Apply(manager "api", expected resourceVersion)
    S-->>ZC: watch event
    ZC->>ZC: mark the kind changed, rebuild after 50 ms
    ZC->>ZC: builders: zones view, policy view (+ the compiled lists)
    ZC->>ZC: snapshot.Store(new) (atomic pointer)
    P->>ZC: next query loads the new snapshot
    Note over ZC: a builder that fails keeps its previous view<br/>and raises Degraded{BuildFailed}

The file loader takes the same path with manager file:<path>: it reads objects: in kudurru.yaml and every objects.d/*.yaml, applies them, deletes the objects that its files no longer declare, and re-applies on a file change (fsnotify, 500 ms debounce) and every 60 s to repair drift. Errors in a file become diagnoses; they never stop the server.

The snapshot's life

stateDiagram-v2
    [*] --> Unsynced: process start
    Unsynced --> Live: every watch has synced
    Unsynced --> FromSnapshot: kube only: no sync within 15 s,<br/>local.db loaded (same cluster ID)
    FromSnapshot --> Live: informers sync
    Live --> Live: store events → rebuild → swap
    note right of Unsynced
        primary zones: SERVFAIL
        (never NXDOMAIN)
    end note
    note right of FromSnapshot
        Degraded ServingFromSnapshot.
        Ready if the copy is ≤ 24 h old;
        older: primary zones SERVFAIL,
        forwarding continues
    end note

In Kubernetes mode each node writes the complete object set to local.db after its first sync and then at most every 30 s when something changed. A pod that restarts while the apiserver is down therefore still answers: forwarded names, cached names, and primary zones from the copy.

The leader

One election covers all leader-only work. The runner starts the leader-only loops under a "term" context when the node wins, and cancels them when it loses. Losing the lease never stops the process: the DNS listeners, the API and the snapshot keep running, which is exactly what you want during an apiserver outage.

stateDiagram-v2
    direction LR
    [*] --> Follower
    Follower --> Leader: wins the Lease (kube)<br/>or at once (sqlite)
    Leader --> Follower: lease not renewed within 10 s
    state Leader {
        [*] --> running
        running: controller, notifier, list fetcher,<br/>unblock janitor, allow/deny sync, audit prune
        running --> running: a loop that fails restarts<br/>with backoff 1 s .. 30 s
    }
    Leader --> [*]: SIGTERM: lameduck, then release the lease

Leader loops are idempotent and level-triggered: they read the store, compare with the status, and write only the difference. A new leader picks up where the old one stopped without special hand-over.

The controller

One reconciler per kind (DNSZone, DNSRecordSet, BlockList, ClientGroup, Policy), each with its own watch and a rate-limited work queue, writing status only when it differs; a full re-list every 10 minutes.

Events (SerialBumped, Duplicated, OutsideZone, …) go to the log only; the Kubernetes Events sink exists but is not wired yet.

Settings

Settings are not store objects. Each key in the metadata table (internal/config/schema.go) has a scope and flags:

flowchart LR
    D["defaults"] --> N["kudurru.yaml<br/>(node file)"] --> SF["settings file<br/>(cluster keys only)"] --> ENV["KUDURRU_SECTION__KEY<br/>and *_FILE"] --> VAL["validate"] --> RUN(["effective config"])

An environment variable wins and locks the key in the dashboard. In Kubernetes the cluster settings are the kudurru-settings ConfigMap (the API refuses to write them; edit the ConfigMap or the Helm values); in Docker the API writes the settings file.

sequenceDiagram
    participant SRC as settings file / ConfigMap / PATCH /api/v1/settings
    participant W as settings watcher (every node)
    participant R as reload subscribers
    SRC->>W: change (fsnotify + 10 s poll)
    W->>W: merge cluster-scope keys only
    W->>R: log level, upstream groups (swap + cache flush),<br/>cache, blocking, block page, rate limiter,<br/>query log privacy, edge cache flush

Not built yet

The follower (v0.2) and raft (v1.0) drivers, the cluster.role: static read-only mode for compose twins, scatter-gather of health and stats across nodes (the node list is already gathered), and a roles: setting. See the design record §4 for the plan.