Vol. 01 — Infrastructure Engineering — MMXXVI

Backends built
to be depended on.

Three internal-platform services — the kind other engineers build on top of. Each proves a distinct discipline of reliability or correctness, not CRUD, on one shared TypeScript spine. Every guarantee tested against real dependencies; every edge case put through an adversarial review.

Services
03
Packages
07
Tests passing
161
Gates green
100%
  1. 01 AuditTrail Append-only, tamper-evident hash-chained audit log
  2. 02 FlagForge Deterministic percentage rollout · server/SDK parity
  3. 03 NotifyCore At-least-once delivery · durable dead-letter queue
§ 01

AuditTrail

Tamper-evidence

An append-only, tamper-evident audit log. Every record is hash-chained to its predecessor like a ledger — rewrite history and verification breaks at the exact row.

  1. Each row stores sha256(prev_hash ‖ canonical content); /verify walks the chain and pinpoints the first break.
  2. Concurrent appends are serialized by a pg_advisory_xact_lock — the chain cannot fork, while reads stay fully concurrent.
  3. Append-only is enforced three ways: no mutate paths, revoked grants on a least-privilege role, and a trigger blocking UPDATE / DELETE / TRUNCATE.
GENESIS0x0000 RECORD 01prev → hash RECORD 02prev → hash RECORD 03TAMPERED GET /verify ✗ break at id 3 reason: content_tampered
Fig. 1 — A rewrite of any record breaks the chain at that row.
§ 02

FlagForge

Deterministic rollout

Feature flags with deterministic percentage rollout, targeting, and a client SDK that evaluates locally using the same engine as the server.

  1. Bucketing is hash(salt ‖ key) → [0,1), depending only on the key — never the weights — so rollouts are monotonic: 10% → 20% only ever adds users. Uniformity verified over 100k keys.
  2. The pure evaluation engine is shared verbatim by server and SDK — identical decisions, zero drift.
  3. The Redis cache is version-guarded by a Lua version floor; a stale reader can never serve an archived flag as live. Redis down → transparent Postgres fallback.
hash(salt ‖ user-42) = 0.173 0.0 1.0 ON · 20% OFF · 80% user-42 → served: variation "on" redis · SET only if version ≥ floor → archived flag never served live
Fig. 2 — Buckets depend on the key alone, so widening the band only adds users.
§ 03

NotifyCore

Durable delivery

Notification delivery on a BullMQ worker — at-least-once with a durable dead-letter queue, idempotency, retries with backoff, and timezone-aware quiet hours.

  1. An atomic claim (UPDATE … WHERE status claimable RETURNING) means two workers can never both deliver; a lease reclaims a crashed worker's work.
  2. The dead-letter transition is DB-authoritative, written before control returns — not a fire-and-forget event — so it survives crashes. A reconciler re-enqueues anything orphaned.
  3. Idempotent enqueue (ON CONFLICT) plus a stable deliveryId handed to providers for downstream dedup.
ENQUEUEidempotent QUEUE WORKERatomic claimUPDATE…RETURNING PROVIDERdeliveryId retry · backoff ×N DEAD-LETTERstatus = dead exhausted POST /dlq/replay replay → re-enqueued, claimed again
Fig. 3 — Exhausted work dead-letters durably and can be replayed.
§ 04

One platform,
not three apps.

packages/shared — the spine

Every service is assembled from the same primitives. buildBaseServer() wires Fastify with the Zod type provider so a single schema drives validation, response serialization, and OpenAPI at once — plus problem+json errors, constant-time API-key auth, and health. createPool() and a fail-fast loadEnv() round it out.

The pure-core pattern

The hardest logic is factored into I/O-free packages, exhaustively unit-tested, and shared verbatim where it matters. flagforge-core's engine runs in the server and the SDK; notifycore-core's quiet-hours math is testable without a clock.

Tested on real dependencies

Integration suites run against real Postgres and Redis in Docker — not mocks — including a live BullMQ worker draining a queue and a killed-Redis degradation check.

Honest threat models

Each service documents guarantees and their boundaries: external anchoring vs. KMS signing, the eventually-consistent bulk-eval snapshot, at-least-once delivery and the transactional-outbox hardening left as future work. Reliability claims you can actually trust.

shared spine audittrail flagforge notifycore + sdk pure cores
§ 05

Colophon & process.

The correctness-critical cores — the hash chain, the bucketing engine, the quiet-hours math — were authored and verified directly. Well-scoped implementation was delegated against those fixed contracts. Then an independent model ran an adversarial review of each service before its gates closed.

That pass earned its keep: it surfaced a cache-aside race that could serve an archived flag as live, a non-durable dead-letter path, and a mixed-type comparison footgun — each fixed and now pinned by a regression test.

Set in Fraunces & IBM Plex. Hand-built, no framework. Read the source ↗

Language
TypeScript · tsx · Node 24
HTTP / API
Fastify 5 · Zod · OpenAPI
Data
PostgreSQL · Drizzle ORM
Cache / Queue
Redis · ioredis · BullMQ
Testing
Vitest · real-dependency integration
Infra
Docker Compose · npm workspaces