Infrastruktur-Monitoring-System/ROADMAP.md

# SoftVisor Infrastructure Monitoring System – Roadmap

Status: v1.5 (2026-09-02) – all five milestones delivered, test instance on port 3333

This document is derived from `CLAUDE.md`. It breaks the project into work packages (WPs),
fixes the technical decisions that are not dictated by `CLAUDE.md`, and lists the open
questions that must be answered before the affected WP starts.

---

## 1. Analysis of the requirements

### 1.1 What we are building

A single web application that monitors and manages one Debian root server which hosts a
Kubernetes cluster with Gitea (and its PostgreSQL database). Four functional areas:

| # | Area | Core capability |
|---|------|-----------------|
| 1 | Update management | Show OS + installed packages/apps with versions; trigger updates |
| 2 | Vulnerability management | Show CVEs for OS, packages, apps, pods, images; e-mail on new findings |
| 3 | Backup management | Define backup strategies (source, interval, target SMB/FTP); run them |
| 4 | User management | Admin creates/manages users; modern auth |

### 1.2 Constraints from CLAUDE.md

- Backend and web server: **Rust**
- Frontend: **Vue.js + Tailwind CSS**
- Clean architecture, pragmatic, no over-engineering, short consistent code
- Commit and push after every feature
- Modern authentication and, where needed, modern encryption
- Ask for clarification when unsure
- **TDD is mandatory:** for every feature, plan and implement the tests (unit, integration
  where applicable, UI) *before* the feature code. Red → green → refactor.

### 1.3 Key observations that shape the architecture

1. **The app must execute privileged actions on the host** (apt, kubectl, reading volumes).
   The safest pragmatic option is to run the backend *on the server itself* as a systemd
   service (or as a pod with host access) and talk to `apt`, the Kubernetes API and the
   filesystem locally. No remote SSH agent layer is needed for a single server.
2. **Backups are application-level, not cluster-level.** Gitea data (repositories, LFS,
   attachments on its PVC) and PostgreSQL (`pg_dump`) are backed up separately. Kubernetes
   manifests can optionally be exported as a third source. Velero or full-cluster snapshots
   are explicitly *out of scope*.
3. **Vulnerability data needs an external scanner.** Writing a CVE matcher ourselves is
   over-engineering. **Trivy** covers all requested targets (Debian packages via
   `trivy rootfs /`, container images, Kubernetes resources) with one tool and JSON output.
4. **Single-tenant, few users.** A local user store with strong password hashing and
   short-lived JWT + refresh tokens is sufficient. OIDC/SSO is a possible later addition,
   not a requirement.
5. **Scheduling is a cross-cutting concern.** Package scans, vulnerability scans and backups
   all run on intervals. One internal scheduler (cron-expressions, persisted state) serves
   all three.
6. **TDD drives the design.** Host commands (apt, kubectl, trivy, smbclient) sit behind
   traits so use cases can be unit-tested with fakes. Every WP starts by writing its tests
   against the planned API/UI, which also fixes the interface before implementation.

---

## 2. Technical decisions

| Topic | Decision | Rationale |
|-------|----------|-----------|
| Web framework | `axum` + `tokio` | Mature, minimal, tower middleware ecosystem |
| Database | SQLite via `sqlx` (compile-time checked queries, migrations) | Single-server app; zero ops; easy backup of the app itself. Switch to Postgres only if ever needed |
| Auth | Argon2id password hashing, JWT access token (15 min) + rotating refresh token (HttpOnly cookie), optional TOTP 2FA in a later WP | Modern, stateless API auth, no external IdP needed |
| Secrets at rest (SMB/FTP credentials, SMTP password) | AES-256-GCM with a master key from env/file | Required by "modern encryption" for stored credentials |
| Package inventory | `apt list --installed` / `dpkg-query` + `apt-get -s upgrade` for available versions; `/etc/os-release` for OS | No extra deps; exact data the OS has |
| Package updates | `apt-get install --only-upgrade <pkg>` / `apt-get dist-upgrade` executed via a narrowly scoped `sudoers` rule; live log streamed to UI | Least privilege, auditable |
| Kubernetes access | `kube-rs` with in-cluster or kubeconfig auth | Native Rust client, typed, watch support |
| Vulnerability scanning | Trivy CLI (JSON output), invoked by the backend | One tool for OS, images, k8s; well maintained DB |
| E-mail | `lettre` (SMTP, STARTTLS/TLS) | Standard Rust mail crate |
| Backup execution | Gitea: tar of PVC data (via `kubectl exec` or hostPath); Postgres: `pg_dump` via `kubectl exec`; optional: manifest export. Compress + optional AES-GCM encrypt, then upload | Application-consistent, restorable without the cluster |
| Backup targets | SMB via `smbclient`/mount, FTP/FTPS via `suppaftp` | Both requested; use proven tooling rather than reimplementing SMB |
| Scheduler | In-process `tokio-cron-scheduler`, jobs persisted in SQLite | One scheduler for scans + backups |
| Frontend | Vue 3 (Composition API, `<script setup>`), Vite, TypeScript, Tailwind, Pinia, Vue Router | Requested stack; TypeScript keeps code short and safe |
| API style | REST + JSON, OpenAPI generated with `utoipa` | Simple; typed client for the frontend |
| Testing | Backend: `cargo test` unit tests per crate, integration tests with `axum` test client + in-memory SQLite, fakes for host adapters. Frontend: Vitest + Vue Test Utils for components/stores, Playwright for UI/E2E against the dev server | Required by the TDD constraint; fast feedback, no real host needed |
| Packaging | One static Rust binary serving the built SPA + API; systemd unit; optional Dockerfile | Simplest deployment on one server |

### 2.1 Clean architecture layout (Rust workspace)

```
backend/
  crates/
    domain/        # entities, value objects, repository + gateway traits (no deps on IO)
    application/   # use cases / services, orchestrate domain + ports
    infrastructure/# sqlx repos, apt/kubectl/trivy adapters, smtp, smb/ftp, scheduler
    api/           # axum handlers, DTOs, auth middleware, OpenAPI, binary entry point
frontend/          # Vue app
deploy/            # systemd unit, sudoers snippet, Dockerfile, example .env
```

Rule of thumb: `domain` has no I/O crates, `application` depends only on `domain`,
`infrastructure` implements the traits, `api` wires everything. Keep it to these four
crates; do not split further unless a crate exceeds a few thousand lines.

---

## 3. Work packages

Effort is a rough size for planning: S (≤1 day), M (2–3 days), L (4–6 days).
Each WP follows the same TDD sequence:

1. **Plan tests** – list the unit tests (use cases, domain rules), integration tests (API
   routes, adapters where a real dependency can be faked) and UI tests (Vitest components,
   Playwright flows) for the feature; commit this list as test stubs.
2. **Implement tests** – write the failing tests first; they define the API and UI contract.
3. **Implement feature** – smallest code that makes the tests pass, then refactor.
4. **Finish** – all tests green, feature committed and pushed, `ROADMAP.md` status updated.

### Phase 0 – Foundation

#### WP-00 Project skeleton (S)
- Cargo workspace with the four crates, `axum` hello endpoint, `/healthz`
- Vue 3 + Vite + Tailwind + TypeScript + Pinia + Router scaffold
- SQLite + `sqlx` migrations setup, config loading from env / `.env`
- `Makefile`: `dev`, `build`, `test`, `lint` (clippy, rustfmt, eslint, prettier)
- CI: GitHub/Gitea Actions running lint + test
- Test harness: `cargo test` layout per crate, integration test helper (test app + temp
  SQLite), Vitest + Vue Test Utils, Playwright with a `make test-ui` target; one sample test
  of each kind runs in CI
- README with dev setup and the TDD workflow
- **Done when:** `make dev` serves frontend + backend, `make test` runs unit, integration
  and UI tests, CI green

#### WP-01 Authentication & user management (M)
- Migrations: `users` (id, email, display_name, password_hash, role, is_active, created_at),
  `refresh_tokens`
- Argon2id hashing, login → JWT access + HttpOnly refresh cookie, refresh rotation, logout
- Roles: `admin`, `user` (viewer/operator). Only `admin` may manage users.
- First-run bootstrap: create initial admin from env or CLI command
- Admin UI: list/create/edit/deactivate users, reset password
- Login page, route guards, auth store, API client with auto-refresh
- Rate limiting on `/auth/login`, audit log table for auth events
- **Done when:** admin can log in, create a user, that user can log in with restricted rights

#### WP-02 Application shell & cross-cutting infra (S)
- Layout: sidebar navigation (Dashboard, Updates, Vulnerabilities, Backups, Users, Settings)
- Global error handling, toast notifications, loading states, empty states
- Settings model + encrypted secret storage (AES-256-GCM, master key from env)
- Scheduler service skeleton (persisted jobs, run history table, manual "run now")
- SMTP settings + "send test mail" via `lettre`
- **Done when:** a dummy scheduled job runs on interval and its run is visible in the UI

### Phase 1 – Update management

#### WP-10 OS & package inventory (M)
- Adapter: parse `/etc/os-release`, `dpkg-query -W`, `apt-get -s upgrade` → `Package { name, installed, candidate, source, is_security }`
- Persist snapshots; scheduled refresh (default hourly) + manual refresh
- UI: OS card (version, kernel, uptime, reboot-required flag), searchable/sortable package
  table with "upgradable" filter and drift indicator (installed vs candidate)
- **Done when:** the UI shows the real package list of the server with available versions

#### WP-11 Package & OS updates (M)
- `sudoers` snippet limiting the service user to `apt-get update`, `apt-get install --only-upgrade`, `apt-get dist-upgrade`, `needrestart`/reboot check
- Use case: update single package, update selected packages, update all; runs as a job
  with streamed log (SSE/WebSocket) and result stored in `job_runs`
- Only `admin` may trigger updates; confirmation dialog in UI
- Track "reboot required" and show banner
- **Done when:** a package can be updated from the UI and the log is visible live

#### WP-12 Kubernetes overview (M)
- `kube-rs` client; list namespaces, deployments/statefulsets, pods, images (with tags/digests),
  node version, PVCs
- Show Gitea + Postgres workloads with image version vs. latest upstream tag (upstream check
  via container registry tag list; best effort)
- Actions (admin): restart rollout, scale, change image tag (= application update)
- **Done when:** cluster state is visible and a rollout restart works from the UI

### Phase 2 – Vulnerability management

#### WP-20 Trivy integration & scan model (M)
- Ensure Trivy present (document install; check on startup)
- Adapter runs `trivy rootfs --format json /` (OS + packages), `trivy image <ref>` for every
  image found in WP-12, `trivy k8s` for cluster misconfig (optional)
- Domain: `Finding { cve_id, severity, package, installed, fixed_version, target, first_seen, last_seen, status }`
- Scheduled scan (default daily) + manual; store results, compute *new* findings by diffing
  with previous scan
- **Done when:** a scan populates findings for OS and all images

#### WP-21 Vulnerability UI & notifications (M)
- Dashboard widget: counts per severity, trend since last scan
- Findings table: filter by severity/target/status, detail drawer with description and links
- Status handling: `open`, `acknowledged`, `fixed` (auto when no longer detected)
- E-mail notification on new findings (configurable min severity, recipients, digest per scan)
- Link from a finding to the package update (WP-11) when a fix version exists
- **Done when:** new CVEs after a scan produce an e-mail and appear as "new" in the UI

### Phase 3 – Backup management

#### WP-30 Backup targets (S)
- Target model: `SMB { host, share, path, user, password }`, `FTP { host, port, tls, user, password, path }`
- Credentials encrypted at rest (WP-02)
- Adapters: SMB via `smbclient` (or mount), FTP/FTPS via `suppaftp`; "test connection" action
- UI: CRUD for targets
- **Done when:** a target can be created and its connection tested

#### WP-31 Backup sources & strategies (M)
- Source types: `GiteaData` (PVC content), `PostgresDump` (pg_dump via `kubectl exec`),
  `KubernetesManifests` (namespace export), `HostPath` (generic directory)
- Strategy: name, source, schedule (cron), target, retention (keep last N), compression,
  optional encryption (AES-256-GCM with a per-strategy passphrase)
- Scheduler jobs created/updated from strategies
- UI: strategy wizard (source → schedule → target → retention)
- **Done when:** a strategy can be created and produces a scheduled job

#### WP-32 Backup execution, history & restore hints (M)
- Run pipeline: snapshot source → archive → compress → (encrypt) → upload → verify size/checksum → apply retention on target
- Run history with status, duration, size, log; failure e-mail
- Manual "run now"; download of a listing from the target
- Document restore procedure per source type (`docs/restore.md`); optional restore for
  Postgres dump
- **Done when:** Gitea data and Postgres dump land on the SMB/FTP target on schedule

### Phase 4 – Hardening & release

#### WP-40 Dashboard & polish (S)
- Landing dashboard: OS version, upgradable count, vuln counts, last backups, failed jobs
- Consistent empty/error states, responsive layout, dark mode (Tailwind)

#### WP-41 Security hardening (S)
- Security headers, CSRF protection for cookie auth, CSP
- Optional TOTP 2FA
- Dependency audit (`cargo audit`, `npm audit`) in CI
- Review `sudoers` scope and file permissions

#### WP-42 Deployment & operations (S)
- systemd unit, service user, `sudoers` snippet, log rotation
- Installer script / Ansible-free shell script for the Debian host
- Backup of the app's own SQLite DB (as a built-in `HostPath` strategy)
- `docs/`: install, upgrade, restore, architecture overview

---

## 4. Delivery order and dependencies

```
WP-00 → WP-01 → WP-02 ─┬─→ WP-10 → WP-11
                       ├─→ WP-12 ──┐
                       │           ├─→ WP-20 → WP-21
                       │  (WP-10) ─┘
                       └─→ WP-30 → WP-31 → WP-32
                                            ↓
                                  WP-40 → WP-41 → WP-42
```

Phases 1, 2 and 3 are independent of each other after Phase 0 and can be reordered if
priorities change. Recommended order is as listed: updates first (quickest visible value),
then vulnerabilities (builds on the inventory), then backups.

Rough total: ~30–40 working days.

---

## 4a. Milestones

### Milestone 1 – Basis, authentication and user management

**Goal:** a deployable application skeleton in which an admin can log in, manage users, and
a regular user can log in with restricted rights. Everything else is navigation placeholders.

| WP | Scope in this milestone |
|----|-------------------------|
| WP-00 | Full scope: workspace, frontend scaffold, SQLite + migrations, `Makefile`, CI, test harness (cargo test, Vitest, Playwright) |
| WP-01 | Full scope: Argon2id, JWT + rotating refresh cookie, roles, bootstrap admin, user CRUD UI, login page, route guards, rate limiting, auth audit log |
| WP-02 (partial) | Only the application shell: sidebar navigation with placeholder pages, global error handling, toasts, loading/empty states. Secret storage, scheduler and SMTP stay in Milestone 2 |

**Deliverables**

- `make dev` starts backend + frontend; `make test` runs unit, integration and UI tests; CI green
- Login / logout / token refresh work end to end (Playwright flow)
- Admin: list, create, edit, deactivate users, reset password
- Non-admin: can log in, sees the shell, gets 403 on admin routes (API and UI)
- Initial admin created on first start from env or CLI
- OpenAPI spec for `/auth/*` and `/users/*`
- README with setup, TDD workflow and how to create the first admin

**Test plan (written first, per TDD)**

- Unit: password hashing/verification, JWT issue/verify/expiry, refresh-token rotation and
  reuse detection, role checks, user use cases (create, deactivate, reset password, no
  self-deactivation of last admin)
- Integration: `POST /auth/login` (success, wrong password, inactive user, rate limit),
  `POST /auth/refresh`, `POST /auth/logout`, `/users` CRUD with admin vs. user token,
  bootstrap admin on empty DB, migrations apply on fresh SQLite
- UI: Vitest for login form, auth store (auto-refresh on 401), user table/form components;
  Playwright for login → user management → logout, and non-admin denied access

**Acceptance:** all deliverables done, all tests green, work committed and pushed, status
table updated. Open questions 1, 9 and 10 (section 5) should be answered before starting;
if not, the listed assumptions apply.

### Milestone 2 – Update management (delivered 2026-09-02)
WP-02 remainder, WP-10, WP-11, WP-12. Verified against the real host: 430 packages, microk8s overview with 16 workloads.

### Milestone 3 – Vulnerability management (delivered 2026-09-02)
WP-20, WP-21. Trivy is installed by the deploy script.

### Milestone 4 – Backup management (delivered 2026-09-02)
WP-30, WP-31, WP-32. Restore procedure in `docs/restore.md`.

### Milestone 5 – Hardening and release (delivered 2026-09-02)
WP-40, WP-41, WP-42. Open items: TOTP 2FA, dark mode, running as a non-root service user with a scoped sudoers file, upstream image tag check, backup failure mails.

---

## 5. Open questions (answer before the affected WP)

The server is reachable via `ssh softvisor` (as root). Findings from the inspection on
2026-09-02, which answer questions 2, 3, 4 (partly), 6 and 10:

- Debian 12 (bookworm), kernel 6.1, ~433 dpkg packages, snaps: `microk8s`, `core20`, `snapd`
- Kubernetes: **microk8s v1.32** (snap); kubectl via `/snap/bin/microk8s kubectl`;
  kubeconfig at `/var/snap/microk8s/current/credentials/client.config` (group `microk8s`)
- Namespaces: `gitea`, `cert-manager`, `ingress`, `metallb-system`, `inlets`
- Gitea: `deployment/gitea` in ns `gitea`, PVC `gitea-shared-storage` (10Gi, hostpath);
  Postgres: `statefulset/gitea-postgresql`, PVC `data-gitea-postgresql-0`; Valkey
  `statefulset/gitea-valkey-primary`. Old unused PVCs from a former HA setup exist.
  All PVCs use `microk8s-hostpath`, so backups can read data from the host filesystem.
- Tools present: `smbclient`, `helm` (needs kubeconfig); **Trivy not installed**
- Git remote is the company Gitea (`git.dev.softvisor.de`) → CI uses Gitea Actions

| # | Question | Affects | Assumption if unanswered |
|---|----------|---------|--------------------------|
| 1 | Does the backend run directly on the Debian host (systemd) or inside the cluster? | WP-00, WP-11, WP-42 | Directly on the host as systemd service |
| 2 | ~~Kubernetes distribution?~~ **Answered:** microk8s 1.32, kubeconfig `/var/snap/microk8s/current/credentials/client.config` | WP-12 | – |
| 3 | ~~How is Gitea deployed?~~ **Answered:** ns `gitea`, `deployment/gitea` + PVC `gitea-shared-storage`, `statefulset/gitea-postgresql` + PVC `data-gitea-postgresql-0`, hostpath storage | WP-12, WP-31 | – |
| 4 | Is Trivy acceptable as external dependency? It is **not yet installed** on the host | WP-20 | Yes, installed in WP-20 |
| 5 | SMTP relay available for notifications? Sender address? | WP-02, WP-21 | Yes, configured via settings UI |
| 6 | ~~Non-apt installs?~~ **Answered:** snaps (`microk8s`, `core20`, `snapd`) and `/usr/local/bin/helm`; snaps will be inventoried in WP-10 | WP-10 | – |
| 7 | Is unattended OS major-version upgrade (e.g. Debian 12 → 13) in scope, or only `apt` upgrades within a release? | WP-11 | Only in-release upgrades; major upgrade is documented, not automated |
| 8 | Backup encryption required, or is transport encryption to the target enough? | WP-31 | Optional per strategy, off by default |
| 9 | Should non-admin users be pure viewers or also able to trigger scans/backups? | WP-01 | Viewers only; all mutating actions are admin |
| 10 | ~~Git hosting?~~ **Answered:** company Gitea → Gitea Actions (`.gitea/workflows`) | WP-00 | – |

---

## 6. Definition of done (every WP)

- Code follows the layer rules in section 2.1; no I/O in `domain`
- Tests were written *before* the implementation (TDD): unit tests for use cases and domain
  rules, integration tests for each new API route and adapter, UI tests (Vitest for
  components/stores, Playwright for the user flow)
- `cargo clippy -D warnings`, `cargo fmt`, `eslint`, `prettier` clean
- API documented in OpenAPI; UI reachable through navigation
- Feature committed with a descriptive message and pushed
- This roadmap updated (status column below)

## 7. Status

| WP | Milestone | Status | Notes |
|----|-----------|--------|-------|
| WP-00 | M1 | done | 2026-09-02 |
| WP-01 | M1 | done | 2026-09-02 |
| WP-02 | M1 (shell) / M2 (rest) | done | shell 2026-09-02; encrypted settings, SMTP, job runner, scheduler 2026-09-02 |
| WP-10 | M2 | done | 2026-09-02; snaps inventoried, no upstream check |
| WP-11 | M2 | done | 2026-09-02; runs as root via systemd, sudoers scoping deferred to WP-41; log via polling |
| WP-12 | M2 | done | 2026-09-02; overview, restart, scale, set image; upstream tag check not done |
| WP-20 | M3 | done | 2026-09-02; Trivy rootfs + image scans, diff with first/last seen, fixed detection |
| WP-21 | M3 | done | 2026-09-02; findings UI, acknowledge, mail digest with severity threshold; dashboard widget in WP-40 |
| WP-30 | M4 | done | 2026-09-02; SMB via smbclient, FTP/FTPS via curl, credentials encrypted |
| WP-31 | M4 | done | 2026-09-02; sources: PVC hostpath, pg_dumpall, manifests, host dir; openssl encryption |
| WP-32 | M4 | done | 2026-09-02; upload verify, retention, records; docs/restore.md; failure mail not yet |
| WP-40 | M5 | done | 2026-09-02; dashboard with tiles for all areas; dark mode not done |
| WP-41 | M5 | done | 2026-09-02; security headers, CSP, body limit, audits in CI; TOTP 2FA and sudoers scoping not done |
| WP-42 | M5 | done | 2026-09-02; install.sh, systemd unit, docs/install.md, docs/architecture.md |