Services
Analytics
The analytics service turns stored pipeline data into insights about traffic and detector behavior. It summarizes total requests, scoring outcomes, risk levels, recommended actions, score trends, and baseline usage.
Data extraction and mart construction
A Python extractor service copies events, decisions, and baseline runs from Postgres into Parquet snapshots.
dbt is used to aid in data modeling workflows by organizing the analytical SQL queries and the tests that validate their results. DuckDB executes these models and builds the final mart, which contains a daily anomaly summary for each analyzed source.
After the models pass their checks, the mart is exported as a static Parquet file for the showcase. The browser loads this file into DuckDB-Wasm, allowing the dashboard to run SQL queries directly in the browser without a separate analytics API.
Output structure
The final mart contains daily summaries of:
| Category | Daily summary contents |
|---|---|
| Traffic | Total requests, distinct visitors and URIs, 4xx responses, and referrer activity. |
| Evaluation | Evaluated, scored, no-baseline, and insufficient-history windows. |
| Risk | Low, medium, high, and critical risk levels assigned to scored windows. |
| Actions | Allow, log, challenge, and block recommendations from the active policy. |
| Scores | Average, p90, p99, and maximum anomaly scores. |
| Baseline usage | When each baseline was used and how many windows it evaluated. |
The dashboard combines these daily summaries across the selected source and date range.
Orchestration
Apache Airflow orchestrates the analytics service’s weekly workflow, which extracts operational data, builds and tests the daily-risk mart, and produces the static Parquet artifact consumed by the showcase.
Extract
Copies events, decisions, and baseline runs from Postgres into raw Parquet snapshots.
Build and test
Uses dbt and DuckDB to build the daily source-risk mart and validate its results.
Export
Writes the completed mart as the static Parquet artifact used by the showcase.
Workflow
The complete analytics flow, from operational data extraction to browser-side querying, is shown below.