Reporting engine (redesign)

Why

The webapp’s report generation was incomplete: Excel worked (reusing the legacy ExcelReportGenerator), but PDF and DOCX were stubs — generate_pdf_report / generate_docx_report in reports_enhanced.py wrote a three-line text file with the wrong extension, so users downloaded a “Word” or “PDF” file that was actually plain text. There were also no charts anywhere, an Excel data-mapping bug, and three divergent report codebases (output/formatters/, output/old/, standalone scripts).

Meanwhile, real, branded generators already exist in src/vulnmgmt/output/formatters/ (python-docx with logo/watermark, WeasyPrint PDF, openpyxl Excel) — they were just never wired into the webapp, and the webapp’s Python env lacked python-docx / weasyprint / jinja2.

The architecture: one IR → many renderers

Adopt a single Report intermediate representation (IR) — a Pydantic model describing semantics, not layout — persisted in Mongo. Every output format is a thin renderer over that IR. This is the AI-native “structured content model → multi-renderer” pattern, made deterministic, testable, and brand-guaranteed.

MongoDB ──► ReportBuilder ──► Report IR (Pydantic, stored in db.reports)
                                   │
    ┌──────────────┬──────────────┼───────────────┬────────────────┐
 to_xlsx        to_docx         to_pdf         to_web_json      (chat→IR)
(XlsxWriter)  (python-docx)  (docx→Gotenberg)  (FastAPI)     (Claude tool call)
    │             │           /LibreOffice)       │
 native chart  matplotlib PNG   reuses DOCX    ECharts / flint-chart

Because the IR is stored, any format can be re-rendered later, and adding a new output (HTML, PPTX) is a new renderer — not a new pipeline. Branding is a brand field selecting a template + palette + chart theme.

The IR (shape)

class ChartSpec(Block):        # Flint-inspired semantic chart
    chart: Literal["bar","line","pie","area"]
    title: str; data: list[dict]
    encodings: dict[str, str]          # {"x":"severity","y":"count","color":"severity"}
class Table(Block):    columns: list[str]; rows: list[list]; conditional: list[Rule] = []
class KeyValues(Block): items: dict[str, str]
class Heading(Block):  level: int; text: str
class Paragraph(Block): text: str; style: str = "Body"

class Section(BaseModel): heading: str; blocks: list[Block]
class Report(BaseModel):
    title: str; subtitle: str; brand: str = "lockstep"
    generated_at: str; sections: list[Section]

Library decisions (from the research)

Concern	Choice	Why
DOCX	python-docx now (reuse the branded generator), docxtpl template-first as fast-follow	Branded header/logo/watermark already implemented; docxtpl lets designers own the template later
XLSX	XlsxWriter	Better native charts + conditional formatting + freeze panes + streaming; reports are write-only
PDF	Render the DOCX, convert via LibreOffice headless / Gotenberg sidecar	One content source + one brand template for Word and PDF; high fidelity. (WeasyPrint only if HTML becomes the canonical report; not ReportLab)
Charts	one ChartSpec → matplotlib PNG (docx/pdf), native chart (xlsx), ECharts/flint-chart (web)	Deterministic server images without a headless browser; live charts in Excel; interactive in the UI
Chat → report	NL prompt → Claude Opus 4.8 tool call returning a schema-validated `Report` → deterministic renderers	LLM chooses sections/tables/charts; never touches brand/formatting

Reject: ReportLab (third layout codebase), docx2pdf (needs Word/COM), per-request soffice forking (use Gotenberg for concurrency).

Docker implications

Python deps (backend image): python-docx, XlsxWriter, matplotlib (MPLBACKEND=Agg), later docxtpl.
PDF: add a gotenberg/gotenberg:8 sidecar container; the backend POSTs the DOCX and gets a PDF back — keeps the heavy LibreOffice runtime out of the app image and handles conversion concurrency. Fonts (Carlito/Caladea/Noto) live in the Gotenberg image.

Build phases

Foundation (this pass) — Report IR + Mongo ReportBuilder (executive, vulnerability, software-inventory, asset-inventory), branding module, charts.py (matplotlib), render_xlsx.py (XlsxWriter + native charts), render_docx.py (python-docx branded + embedded chart PNGs). Wire reports_enhanced.py to the new service (replace the PDF/DOCX stubs, fix the Excel constructor bug), and store the IR in db.reports.
PDF — Gotenberg sidecar + render_pdf.py (DOCX→PDF).
UI — a redesigned Reports page: report-type × format matrix, filters, live preview (web-json + ECharts/flint-chart), correct Lockstep-orange branding.
Chat-to-report — a Claude tool call that emits a Report IR from natural language, rendered by the same deterministic renderers.

Reference

Sample outputs the redesign targets for fidelity live under reports/templates/ and reports/samples/, and the legacy branding intent is captured in the DOCX_* notes under docs/_legacy/.