--- metadata: date: "2026-11-05" description: Agents write the code, and people still carry production signal back to them. What the backward pass looks like in a running pipeline engine, and what it proves before a change ships. title: Self-improving software starts where the data has labels next_page: path: /compare/airflow title: Millstream vs Airflow, Dagster and Prefect --- # Self-improving software starts where the data has labels *5 November 2026* Agents now write much of the code. The forward pass of software, from intent to code, is largely automated. The backward pass, from production signal to the next change, usually still runs by hand: someone reads a dashboard, correlates it with a few tickets, and prompts an agent. Self-improving software closes that loop. It reads its own signals, decides what to change, and checks whether the change worked. Millstream is built around this loop for one kind of software: pipelines that mix code, LLM agents and people. This article shows what the backward pass looks like in a running engine, and what it proves before a change ships. Millstream is pre-release, and the figures below come from its acceptance scenario on synthetic data. --- ## Why pipelines get there first The backward pass needs three things an application rarely has in one place: a record of what happened, a judgement of whether it was right, and a way to trace that judgement to the code responsible. A pipeline has all three as a side effect of running. Every item is persisted and immutable, and every change is an event in one log. When an analyst corrects a score or rejects a lead, that decision is a label written by someone who knows the answer, attached to the exact item the pipeline produced. Past items can be replayed through a changed step. Most software teams would pay a lot for data this clean about their own product, and a pipeline with a review step collects it every day. --- ## The backward pass, part by part Software can be read as a neural network: code as the weights, production signal as the output. Each part of that comparison has a counterpart in Millstream. | In the analogy | In Millstream | | --- | --- | | The loss | Agreement with reviewers, failure rate, cost per approved item, items wrongly dropped | | The gradient | Attribution: an edited field walks the item's lineage up to the step that last changed it | | The weights | Lessons, prompts, thresholds, the graph and code, all part of the revision's content hash | | The update | A proposal: a patch with its rationale, the insights it answers and a validation report | | The validation set | A backtest over past items, and a holdout the drafting agent never sees | Attribution deserves a closer look. Because items are immutable, the comparison between a step's input and output is exact, and pass-through steps are never blamed. A reviewer's edit to a price arrives at the step that wrote the price, so the [lessons proposal](/platform/self-improvement) that follows changes that step and leaves the others alone. --- ## What the analogy leaves out A training loop with no validation set overfits, and the same holds for software that rewrites itself. An agent opening a pull request with a fix and its evidence has done the easy half. Everything hard about self-improvement happens after that point, so Millstream puts its effort there. A proposal has to pass before anyone reads its diff. A backtest replays it on a sample of past items with external effects stubbed, a distilled step is judged on a holdout its drafting agent never saw, and the base revision's evals decide, so a proposal cannot redefine its own success. Evidence is treated as untrusted, since a ticket or a crawled page is written by a stranger: a patch may only touch the files its kind allows, and the published patch is the one whose hash was validated. [Self-improvement](/platform/self-improvement) covers each rule, and how evidence leaves production. --- ## Where the intelligence goes Inference is now cheap enough to read production signal at scale, sampled and under a budget. Millstream still computes its insights with no model at all, because deterministic detection costs nothing, gives the same answer twice and works on a laptop with no API key. On the forward path, an agent is paid for every item, varies between runs and adds latency to every step. So the loop's most useful result is a self-improvement that takes the model out. When an agent step's answers become predictable, Millstream proposes [code that replaces it](/platform/self-improvement), with the agent kept as the fallback for the items the code abstains on and as a check on a small sample of the rest. In the acceptance scenario, on synthetic listings whose answers follow a rule: - **0.58 → 1.00** agreement with reviewers after a lessons proposal - **260 of 270** held-out items covered by the code, all in agreement - **5.2%** of the agent's cost on the next day, audits included A synthetic scenario shows the loop running end to end. On a real pipeline, the figures depend on its data. --- ## What stays with people A self-improving system still has to be told what "better" means, and goals pull against each other. A lower cost can mean more wrong answers, and fewer reviews can mean less to learn from. Choosing the weights and the hard constraints was always the hard part, and it stays with the people who answer for the result. Their time is best spent on judgement. Reading a validated diff and accepting it takes minutes, and someone stays accountable for the change. Autonomy can then grow the way review rules already earn it. A `review.autoApprove` rule approves items while people check a sample, and a falling agreement raises an insight. The same model suits proposals: a class of change earns acceptance without a person through its measured record, and loses it the same way. Every proposal in v1 goes through a person, and policies for low-risk classes come after v1. --- ## Sources Boris Tane, [Self-improving software is inevitable](https://boristane.com/blog/self-improving-software-is-inevitable/), October 2026. --- ## Next [How pipelines learn](/platform/self-improvement) [Agents in production](/docs/production-checklist) [Get early access](/pricing#early-access)