Platform Pipelines Agents Human review Self-improvement Observability Deployment Security Use cases Compare Docs Pricing Roadmap Get early access

Deployment

One binary is the engine, the CLI and a node, with the web UI embedded. It runs the same packages everywhere, and the environment file says where. Each customer gets a deployment of their own, on their infrastructure.

On a laptop

millstream run in a package directory runs the engine and an embedded node on SQLite and serves the API and the UI on 127.0.0.1:7070. There is nothing to configure and no service to install.

millstream run                 # engine, embedded node, API and UI
millstream run --once          # fire the manual triggers, wait for the runs, print a summary, exit
millstream init environment    # millstream.yaml, every key commented out at its default

On a server

The same binary runs with serve against Postgres. The repository ships a Docker Compose file with Postgres, an active engine, a standby engine and two nodes, and systemd units for hosts without containers.

cd deploy/compose
cp .env.example .env           # the proxy's URL, and openssl rand -hex 32 for each secret
docker compose up -d
millstream apply ./property-sourcing --server https://millstream.internal

environment: production
store: { dsn: postgres://db.internal/millstream }
artifacts: { url: s3://millstream-artifacts }
listen: 127.0.0.1:7070
auth: { token: { env: MILLSTREAM_AUTH_TOKEN } }
node: { embedded: true, token: { env: MILLSTREAM_NODE_TOKEN } }
retention: { events: 90d, transcripts: 30d, rollups: 400d }

Every key of the environment file is also a MILLSTREAM_* variable, so Compose, systemd and a shell configure it the same way. Postgres and S3 credentials come from their libraries' own variables, never from the file.

A hot standby

A second millstream serve on the same Postgres store is a hot standby. Leadership is a Postgres advisory lock held on the engine's only write connection, so there is never a second writer, whatever the clocks or the network do. When the active engine dies or loses its session, the standby takes the lock, rebuilds its state from the store and starts dispatching. While it waits, it serves the reads and the UI.

Remote nodes

A node is the same binary on any host. It joins the engines with the deployment's node token over TLS, pulls the deliveries it can run, and follows the active engine across a failover.

MILLSTREAM_NODE_TOKEN=… millstream node --join https://engine:7070,https://engine-standby:7070 --labels gpu=true,zone=eu

Several pipelines, one deployment

A deployment runs many isolated pipelines, each with its own secrets, limits and lifecycle. Dispatch is round-robin across pipelines, so a backlog in one never starves another. apply uploads a package as a new revision, rollback reactivates an earlier one, and runs in flight finish on the revision they started with.

Retention, erasure and backups

Built to be taken apart

The crash suite kills the engine and its nodes at every boundary of every atomic unit, on SQLite and on Postgres, and checks the invariants after each: no item lost, no output published twice, every run settles, and both stores reach the same state. The chaos suite kills active engines, partitions nodes behind a fault-injecting proxy and skews their clocks. The soak runs the Compose deployment under random kills and partitions:

Run Chaos Runs Items Kills Partitions Standby lock Checks
30 minutes every 2 min on average 168 15,837 3 engines 3 nodes 29–30 ms all pass
5 minutes every 20 s on average 27 2,528 2 engines, 4 nodes 1 node 30–31 ms all pass

Packaging

Artefacts

Targets

Requirements

Next

Security Observability Pricing