fasten docs — audit + correlation SDK

Installation

PyPI and npm packages publish at v1.0.0-beta.0 (roadmap). Until then, install from source.

Today — install from source

bash

pip install ./python                        # Python — reference impl
go get github.com/nerdapplabs/fasten-go    # Go (module path, works today)
npm install ./js                            # Node / TypeScript
# C++14: copy cpp/include/fasten.hpp — zero dependencies

v1.0.0-beta.0 — coming soon roadmap

bash

pip install fasten==1.0.0b0                     # Python — reference impl
pip install 'fasten[tui]==1.0.0b0'              # + bundled live TUI
npm install @nerdapplabs/fasten@1.0.0-beta.0    # Node / TypeScript

5-minute quickstart

1 Register audit codes — once, at startup

python

import fasten
from fasten.codes import register, Meta, Severity, RetentionClass

register("user", {
    "USER_CREATED": Meta(
        domain="user", category="account",
        action="create", severity=Severity.INFO,
        description="New user account created", emitter="auth-service",
        retention_class=RetentionClass.LONG,
    ),
    "USER_DELETED": Meta(
        domain="user", category="account",
        action="delete", severity=Severity.WARN,
        description="User account permanently deleted", emitter="auth-service",
        retention_class=RetentionClass.LONG,
    ),
})

2 Init — reads env vars; `FASTEN_AUDIT_DSN` required

.env

FASTEN_AUDIT_DSN=sqlite:///./audit.db   # or postgres://...

python

fasten.init(service_id="auth-service", node_id="host-01")
# reads FASTEN_AUDIT_DSN; raises RuntimeError if unset.
# Audit rows go to durable storage — no silent in-memory fallback.

3 Emit anywhere

python

fasten.emit(code="USER_CREATED", target="u-42",
           actor="admin", detail={"email": "alice@example.com"})

fasten.log.info("signup_complete", user_id="u-42")

Production: env-var driven

.env

FASTEN_SERVICE_ID=auth-service
FASTEN_NODE_ID=host-01
FASTEN_AUDIT_DSN=postgres://user:pw@db:5432/appdb

python

fasten.init()   # reads everything from env — same emit() call

What you actually see

Every emit() and log.* writes one NDJSON line to stdout. Docker's log driver captures and rotates it.

emit("USER_CREATED", …) → stdout

stdout — shape: audit

{
  "shape": "audit",
  "id": "evt-a1b2c3d4e5f6a7b8c9d0",
  "monotonic_seq": 1,
  "timestamp": "2026-04-24T10:23:45.123Z",
  "code": "USER_CREATED",
  "action": "create",
  "severity": "info",
  "service_id": "auth-service",
  "source_node_id": "host-01",
  "actor": "admin",
  "actor_kind": "human",
  "target": "u-42",
  "category": "account",
  "domain": "user",
  "method": "http",
  "request_id": "d4e5f6a1b2c3",
  "detail": {"email": "alice@example.com"}
}

log.info("signup_complete", …) → stdout

stdout — shape: sys

{"shape":"sys","level":"info","event":"signup_complete","request_id":"d4e5f6a1b2c3","service_id":"auth-service","timestamp":"2026-04-24T10:23:45.124Z","user_id":"u-42"}

Both lines share request_id: "d4e5f6a1b2c3" — the join key across all three streams.

Querying back

bash

# All audit rows for one request
curl "http://localhost:8080/api/v1/logs/audit?request_id=d4e5f6a1b2c3"

# Recent warnings in syslog
curl "http://localhost:8080/api/v1/logs/sys?level=warn&limit=50"

# USER_DELETED events in the last 7 days
curl "http://localhost:8080/api/v1/logs/audit?code=USER_DELETED&since=2026-04-17T00:00:00Z"

# Filter by actor + target, paginate
curl "http://localhost:8080/api/v1/logs/audit?actor=admin&target=u-42&limit=20&offset=0"

Audit responses are {"rows": [...], "total": N, "limit": L, "offset": O}. Sys / api responses are {"rows": [...]} (ring buffers, no offset).

Debugging a real incident

Scenario: A pipeline config change was applied at 14:32. By 14:33 the MQTT broker shows message loss.

Without fasten

docker logs gateway | grep 14:32 — hundreds of lines, no actor info
docker logs pipeline-engine | grep 14:32 — different format, different timestamps
Cross-reference manually — which request caused which effect? Unknown.
Ask on Slack "who changed the config?" — 20-minute delay

With fasten — one query, 30 seconds

bash

curl "/api/v1/logs/audit?since=2026-04-24T14:30:00Z&until=2026-04-24T14:35:00Z"

response — 3 rows, same request_id threads all of them

{ "total": 3, "limit": 100, "offset": 0, "rows": [
  { "timestamp": "14:32:11Z", "code": "CONFIG_NODE_UPDATED",
    "actor": "praveen", "actor_kind": "human", "method": "http",
    "target": "pipelines/modbus-01/connection/url",
    "request_id": "req-9f8e7d6c",
    "detail": {"old": "modbus://10.0.1.10", "new": "modbus://10.0.2.10"} },

  { "timestamp": "14:32:12Z", "code": "CONNECTOR_DISCONNECTED",
    "actor": "system", "method": "mqtt", "target": "modbus://10.0.1.10",
    "request_id": "req-9f8e7d6c" },

  { "timestamp": "14:32:14Z", "code": "CONNECTOR_CONNECTED",
    "actor": "system", "method": "mqtt", "target": "modbus://10.0.2.10",
    "request_id": "req-9f8e7d6c" }
] }

Same request_id threads the HTTP config change, MQTT disconnect, and reconnect — across two services and two transports. 30 seconds vs 20 minutes.

The 7 anchors

5 Ws + H + CORRELATION — enforced at the type level. Caller supplies only code, target, and optional detail. Everything else auto-fills.

Anchor	Fields	Auto-filled?
WHO	`actor`, `actor_kind`	from ctx / caller
WHAT	`code`, `action`	`code` from caller; `action` from Meta
WHEN	`timestamp`, `monotonic_seq`	auto
WHERE	`source_node_id`, `service_id`, `tenant_id`	auto from `fasten.init()`
WHOM	`target`, `category`, `domain`	`target` from caller; rest from Meta
HOW	`method` ∈ {http, mqtt, cli, scheduler, ui, agent_tool}	auto from shim / caller
CORRELATION	`request_id`	from context; minted if absent

WHY lives in detail.reason (free text). A policy plugin can enforce it on mutation codes.

Three streams

Stream	Storage	Endpoint	Persistent?
syslog	In-memory ring (10k lines)	`/logs/sys`	No — ring only
API log	Ring + opt-in SQL	`/logs/api`	Opt-in via `FASTEN_API_DSN`
audit	SQL `fasten_audit`	`/logs/audit`	Yes — per-code TTL

All three carry request_id. A ?request_id=<id> query returns rows from all services that carried it.

Shims

Nothing default-on. Import only what your transport uses. Each shim does one thing: read an id from the wire → stash in context → propagate downstream.

Shim	Wire convention	Status
`shim.http`	`X-Request-ID` header (mint if absent)	bundled
`shim.mqtt`	`_req` field inside payload	bundled
`shim.scheduler`	Mints `scheduler-<run_id>` at job start	bundled
`shim.agent_tool`	Agent tool-call context propagation	planned
Custom (gRPC, NATS, …)	10-line pattern: read wire → set ctx	—

Schema

fasten owns one SQLite file (fasten.db). Your product owns its own DB. They share a request_id join key — they never share a file.

storage layout

{DATA_DIR}/fasten.db       ← fasten owns this (created by fasten.init())
  └── fasten_audit          ← one row per emit(); TTL-swept by fasten
  └── fasten_api_log        ← opt-in; one row per HTTP request

{DATA_DIR}/your-app.db   ← your product owns this
  ├── ...your tables...
  └── audit_replay         ← YOUR table; soft-ref to fasten_audit.id
                             holds replay_payload for recovery

fasten_audit columns

Column	Type	Notes
`id`	TEXT PK	`evt-<20-hex>`
`code`	TEXT	e.g. `USER_CREATED`
`actor` / `actor_kind`	TEXT	WHO anchor
`target` / `category` / `domain`	TEXT	WHOM anchor — adopter-defined strings
`service_id` / `source_node_id` / `tenant_id`	TEXT	WHERE anchor — from `init()`
`method`	TEXT	HOW anchor
`request_id`	TEXT	CORRELATION — join key across streams
`timestamp` / `monotonic_seq`	TEXT / INT	WHEN anchor
`detail`	JSON	Adopter-owned free blob; secrets redacted

fasten creates fasten_audit via idempotent CREATE TABLE IF NOT EXISTS on init(). Your product creates audit_replay via its own migrations. Never mix them in the same file.

Security model

fasten is responsible for the row's integrity from emit to insert, and for any surface fasten itself exposes — but never for the security of storage you brought.

Responsibility split

Zone	Owner
At write redaction · schema · registry integrity · SQL safety	fasten
At rest file perms · encryption · backup	You. Perimeter is your DB's existing security — RBAC, KMS, network isolation.
At read authn · authz · CORS · rate limit	You. Mount the reader behind your existing auth (FastAPI `Depends`, gateway, session middleware).
Distribution signed releases · SBOM · CVE disclosure · vuln scanning	fasten

Wire your existing auth

The reader is a FastAPI router. Mount it like any other route, behind whatever auth your app already runs.

python

from fastapi import Depends
from your_app.auth import require_audit_read   # your existing dependency

app.include_router(
    fasten.reader.router(),
    prefix="/api/v1/logs",
    dependencies=[Depends(require_audit_read)],   # ← your check, your rules
)

fasten's reader doesn't add its own check here. Two auth layers fight each other; one well-placed layer is the point.

Reserved headers

Header	Purpose
`X-Fasten-Request-Id`	Correlation id bundled

What stays out of scope

TLS termination — your reverse proxy / load balancer.
Network ACLs — your VPC / firewall / k8s NetworkPolicy.
Key distribution — your secret manager (Vault, k8s Secret, AWS SSM).
DB encryption at rest — your DB layer (e.g. Postgres TDE) or OS-level on the SQLite file.

API mounting

fasten's reader is a mountable router — not a standalone service. Wire it in like any sub-router under whatever prefix you choose.

Python (FastAPI)

python

import fasten
from fastapi import FastAPI
from fasten.shim.http import RequestIDMiddleware, APILogger

fasten.init()                                  # reads FASTEN_AUDIT_DSN etc.
app = FastAPI()
app.add_middleware(RequestIDMiddleware)        # mints / honours X-Request-ID
app.add_middleware(APILogger,
                    skip={"/health", "/metrics"})  # one api row per request
app.include_router(fasten.reader.router(),     # auto-uses fasten.init() globals
                    prefix="/api/v1/logs")

# GET /api/v1/logs/sys?level=warn&limit=50
# GET /api/v1/logs/api?path=/users
# GET /api/v1/logs/audit?actor=admin&target=u-42&offset=0&limit=20

Go (chi)

import (
    fasten    "github.com/nerdapplabs/fasten-go/fasten"
    httpshim "github.com/nerdapplabs/fasten-go/fasten/shim/http"
)

fasten.Init(fasten.Config{})
r := chi.NewRouter()
r.Use(httpshim.RequestID)
r.Mount("/api/v1/logs", fasten.NewReader())
// GET /api/v1/logs/sys  |  /api  |  /audit

Env-var reference

No fasten.yaml. No config daemon. Env-vars only.

Variable	Default	Purpose
`FASTEN_SERVICE_ID`	required	WHERE — service identity
`FASTEN_NODE_ID`	required	WHERE — host/node identity
`FASTEN_TENANT_ID`	unset	WHERE — tenant / org / site (optional)
`FASTEN_AUDIT_DSN`	required	Audit store (`sqlite://` or `postgres://`) — fasten refuses to start without it
`FASTEN_API_DSN`	ring only	Opt-in API-log SQL persistence
`FASTEN_LEVEL`	`info`	Syslog threshold
`FASTEN_REDACT_KEYS`	sensible default	Extra redaction patterns (comma-sep)
`FASTEN_SYS_RING_SIZE`	`10000`	Syslog ring size (lines)
`FASTEN_API_RING_SIZE`	`10000`	API-log ring size (lines)
`FASTEN_AUDIT_RETENTION_CLASS_SHORT`	`30`	Days for `short`-class codes
`FASTEN_AUDIT_RETENTION_CLASS_MEDIUM`	`180`	Days for `medium`-class codes
`FASTEN_AUDIT_RETENTION_CLASS_LONG`	`1095` (3 yr)	Days for `long`-class codes
`FASTEN_RETENTION_SWEEP_INTERVAL`	`6h`	TTL sweep cadence

Retention + PII

Class	Default TTL	Example codes
`short`	30 days	High-volume operational codes
`medium`	180 days	`CONNECTOR_STARTED`, `SCHEDULE_TRIGGERED`
`long`	3 years	`USER_CREATED`, `CONFIG_UPDATED`, `AUTH_LOGIN`

PII — three mechanisms

1. Redaction at emit time — keys matching known patterns (api_key, password, token, authorization) have values replaced with *** before writing anywhere.

python

fasten.emit("USER_CREATED", target="u-42",
           detail={"email": "alice@acme.com", "api_key": "sk-abc123"})
# stored: {"email": "alice@acme.com", "api_key": "***"}

fasten.init(..., extra_redact_keys=["ssn", "credit_card", "dob"])

2. PII class flag — forces retention to short regardless of declared class:

python

Meta(..., pii_in_detail=True)   # overrides → retention_class: short (30 days)

3. TTL sweep — rows beyond their class are deleted on schedule. For GDPR right-to-erasure: delete rows by actor or target, then let TTL sweep clean up the rest.

Operational FAQ

What is the latency overhead?

~0.1–0.5 ms per emit() with SQLite WAL. ~1–5 ms with Postgres. fasten.log.* is ring + stdout only — sub-millisecond. High-volume codes (>100/sec): set high_volume=True in Meta to skip the SQL insert.

What is the memory footprint?

~8 MB at full occupancy: syslog ring (10k × ~500 B) + API log ring (10k × ~300 B). Tune with FASTEN_SYS_RING_SIZE and FASTEN_API_RING_SIZE.

Does SQLite contend under concurrent writes?

WAL mode allows concurrent readers + one writer. Typical audit volumes are well below contention. Mark hot codes high_volume=True or switch to Postgres if you see it.

Does fasten add a thread or process?

No. In-process library. Ring buffer uses a mutex (channel in Go). No goroutines, no threads, no sidecar.

What happens if the DB is unavailable?

The row is written to stdout regardless — Docker / journald captures it.
The SQL insert fails silently in the request path; the row is not lost.
For guaranteed durable SQL persistence, use the outbox variant with a retry queue (planned).

Does fasten work in air-gapped environments?

Yes. Stdout is the primary transport — no network required. SQLite works fully offline. Rows drain upstream when connectivity resumes.

Code evolution + compatibility

Renaming a code

python

# v1
register("user", {"USR_CREATED": Meta(...)})

# v2 — rename; keep old emittable for one release
register("user", {
    "USER_CREATED": Meta(...),
    "USR_CREATED":  Meta(..., declared_unused=True),  # old rows still readable
})

Schema contract

The 7 anchor columns are stable — never change type. detail is yours to evolve.

Identity: id, origin_id, monotonic_seq, timestamp
WHAT: code, action, severity
WHERE: service_id, source_node_id, tenant_id
WHO: actor, actor_kind
WHOM: target, category, domain
HOW: method
CORRELATION: request_id
Free JSON: detail — you own its schema

OTel / trace_id bridge

Pass your OTel trace_id as fasten's request_id — they become the same id. A future transport/otlp plugin will export audit rows as OTel LogRecord entries, letting existing Grafana/Jaeger stacks query fasten rows.

python

from opentelemetry import trace
span = trace.get_current_span()
ctx = fasten.context.set_request_id(
    format(span.get_span_context().trace_id, '032x')
)

Installation

5-minute quickstart

1 Register audit codes — once, at startup

2 Init — reads env vars; FASTEN_AUDIT_DSN required

3 Emit anywhere

Production: env-var driven

What you actually see

emit("USER_CREATED", …) → stdout

log.info("signup_complete", …) → stdout

Querying back

Debugging a real incident

Without fasten

With fasten — one query, 30 seconds

The 7 anchors

Three streams

Shims

Schema

fasten_audit columns

Security model

Responsibility split

Wire your existing auth

Reserved headers

What stays out of scope

API mounting

Python (FastAPI)

Go (chi)

Env-var reference

Retention + PII

PII — three mechanisms

Operational FAQ

What is the latency overhead?

What is the memory footprint?

Does SQLite contend under concurrent writes?

Does fasten add a thread or process?

What happens if the DB is unavailable?

Does fasten work in air-gapped environments?

Code evolution + compatibility

Renaming a code

Schema contract

OTel / trace_id bridge

2 Init — reads env vars; `FASTEN_AUDIT_DSN` required