The Factory Should Not Know the Project

A software factory should know how to manufacture software. It should know how to provision work, route models, control cost, execute changes, collect evidence, recover from failure, and move work through increasingly strong gates. What it should not know is what good means for every project it will ever build.

That boundary is no longer theoretical in factory experiments I am running. The split is encoded directly at the enforcement line. A factory standard reads the factory repository and can fail the factory build. A project-specific standard reads a registered target repository and can fail a build into that target. Policy sits beside both and behaves differently again: it supplies a tunable value rather than something that is violated. The current machine has twenty-five factory standards and six target-specific standards.

One of my current experiments happens to be a game. Nothing in the factory should ever need to learn what a bonk is.

That sentence captures the boundary better than most of the architecture diagrams I could draw. The factory can know how to run a test, invoke a reviewer, record evidence, enforce a standard, or recover from a failed build. The meaning of a bonk, whether the first floor is paced correctly, and why a combo is unreachable belong somewhere else.

That somewhere is the project harness.

So the shape I am converging on has three parts. The factory provides reusable manufacturing capability. The project harness grounds that capability in the particular thing being built. Factory memory carries reusable manufacturing knowledge from one build toward the next.

I have used the word harness more narrowly before, so there is one distinction worth making early. An execution harness surrounds a single worker run: context, tools, permissions, budget, retries, and the controls needed to make an agent useful. A project harness operates at a different scope. It carries the purpose, knowledge, standards, policies, roles, work definitions, and evidence obligations that make the factory capable of building this particular system rather than merely generating software.

The boundary is enforceable

The reusable parts of a factory are substantial. There is execution infrastructure, model and cost control, and a build kernel that knows how to move work through the system. Those mechanisms should not care whether the target is an iOS application, a compliance platform, a game, an internal experiment, or something I have not thought of yet.

The kernel should know how to take an Article, assemble the required context, dispatch work, invoke the appropriate Actors, gather artifacts and evidence, and decide what happens next. Those are manufacturing concerns. They describe how work moves through the factory. They do not describe what the thing being built means.

The game experiment makes the difference obvious. During a play test on September 19, the useful observation was not some generic software quality metric. It was: field empty 75% of floor 1, roughly 8 drops per minute, combos unreachable.

That is valuable knowledge. It is also knowledge the factory should never own. The build kernel does not need to understand the concept of a bonk. The model proxy does not need to understand a combo. The factory does not need to reason about floor-one pacing. Those concepts matter because of this particular experiment, so they belong with its harness.

This is where I think much of the current conversation about agent harnesses stays too close to the individual worker. Tools, skills, prompt construction, and context management all matter, but they mostly answer the question, "How do I make this agent capable of doing work?"

A software factory has to answer a different question: "How does reusable machinery know what matters for this particular build without becoming that build?"

That answer is not another skill. It is the harness around the project.

The harness is mostly knowledge before it becomes machinery

A surprising amount of the project harness begins as text. Purpose, domain knowledge, architecture decisions, craft standards, building standards, actor responsibilities, council charters, acceptance criteria, Articles, and policies are all expressed first through language.

That can make the harness look softer than the machinery underneath it. I think that is exactly backwards. This material is not documentation sitting beside the real system. It is part of the control system.

Every build needs a grounded model of its own world. It needs to know what terms mean, which workflows matter, which systems are authoritative, what architectural decisions have already been made, and which facts the work is not allowed to casually reinterpret. That is knowledge.

A standard does something different. It establishes what must remain true. Knowledge might say that a particular system owns an account balance. A standard might say that every transformation affecting that balance must preserve provenance. Knowledge describes the terrain. Standards constrain what is allowed to happen inside it.

A policy is different again. It is not simply a fact about the world, and it is not necessarily something that can be violated. It is a deliberate operating choice that somebody is allowed to tune. The current merge gate, for example, has retirement thresholds of twenty escape-free merges for a change class and twenty-eight unbroken nightly witness runs. Nobody "violates" twenty-eight. Someone may decide that twenty-eight is too conservative or not conservative enough and change it deliberately. What matters is that the value exists, the authority to move it is clear, and the change leaves a record.

Then there is evidence, which tells us what actually happened and gives us reason to believe the work satisfies the knowledge, standards, and policies around it.

Those distinctions matter:

Knowledge says what is true. A standard says what must remain true. A policy says what is currently allowed or required while remaining deliberately adjustable. Evidence tells us what actually happened.

Collapsing all four into "context" throws away too much of their meaning.

Craft standards travel. Building standards do not always travel.

Some standards sit close to the factory because they represent craft I expect to carry from build to build. Testing discipline, observability, architecture boundaries, security, and recovery do not become irrelevant because the next thing I build happens to be a game.

One of my current projects has 592 tests and an 85 percent coverage gate. Those numbers belong to that repository today, but the underlying obligation can travel. A factory should have opinions about whether software is being manufactured with evidence, whether failures can be observed, and whether the structural boundaries a system claims to have are actually defended.

Other standards only make sense because of the structure being built. A game has pacing, progression, rendering, and playability concerns that would be absurd to encode in a general-purpose build kernel. A system promising reproducibility may need controlled clocks, identifiers, transformations, and inputs.

The distinction is not that factory standards are serious while building standards are optional. Both can have teeth. The distinction is where they apply and what they are allowed to know.

That boundary is already encoded in the newer factory architecture. Factory standards live with the factory and can fail the factory. Target-specific standards live with the registered target and can fail a build into that target. A build carries the standards declared by the target, and an obsolete or unknown declaration fails closed rather than silently disappearing.

The enforcement mechanism can be reusable even when the meaning being enforced is local. That is exactly the boundary I want.

Standards start as language and gain teeth

Writing a standard down is not the same thing as enforcing it. One lesson from the first factory was how easy it is to believe a system has a standard because that standard exists in Markdown. If nothing notices when the rule is broken, the standard is mostly aspirational.

The interesting transition happens when language acquires mechanisms.

Take determinism. It might begin as a craft principle: core logic should not read wall-clock time or mint identifiers inline. At first, an architectural actor may inspect a change and decide whether the principle has been respected. Later, an evaluator can inspect for forbidden patterns. Eventually the repository can have structural guards that simply fail when the wrong code appears in the wrong place.

The principle began as text, passed through judgment, and eventually became a mechanical control. The text does not become useless when the control appears. It still defines the property being protected and explains why the mechanism exists. The mechanism gives that property teeth.

But this does not happen automatically. On September 20, the game experiment's README said there were sixteen levels while the underlying data contained eighteen. Nothing in the system noticed. I found it by hand.

That is a small error, but it exposes a larger problem. If knowledge is part of the harness, knowledge can drift just like code. A standard without enforcement can silently become meaningless. Knowledge without a witness can silently become false.

Standards need enforcement. Important knowledge needs witnesses. Where the system can test whether a claim about itself remains true, it probably should.

Articles bind work to proof

Evidence is why I no longer think of work in the factory as a queue of tasks.

A task says, "Implement authentication." An Article carries a stronger contract. It describes the work, the expected artifact, the relevant constraints, and the evidence owed before the factory can treat the work as complete.

The factory should not merely produce artifacts. It should produce artifacts together with evidence that gives us reason to trust them.

Some of that evidence is mechanical. Tests passed. A mutation caused the expected guard to fail. A schema validated. A threshold held. Other evidence requires judgment. The architecture still makes sense. The behavior matches the domain. The implementation satisfies the intent rather than merely the syntax of the acceptance criteria.

The game experiment gave me a better example than conventional testing. I needed evidence about actual play behavior, so the harness required a gated Game.AI hook and a Playwright video runner. That rig was built in one session. It immediately found a floor-one soft-lock caused by tutorial code running inside the per-frame updateGearEffects(), recreating three canvas-backed objects roughly sixty times a second until Phaser's CanvasPool exhausted and setText crashed.

The software could run. The normal test suite could be green. The experience that mattered was still broken.

That is why tests, evals, adversarial reviews, scenario runs, telemetry, council verdicts, and human approvals increasingly look like members of the same family to me. Operationally they are different. Architecturally they all exist to produce evidence.

An Article is not complete because the code exists. It is complete when the artifact exists and the evidence owed by that Article exists with it.

When generation becomes cheap, another implementation is not particularly scarce. Confidence is. What do we know? Why do we believe it? Which mechanism produced that belief? What observation would cause us to stop believing it?

The factory can provide general machinery for collecting and moving evidence. The project harness decides which evidence matters here and what counts as sufficient.

That makes verification part of production rather than a cleanup phase after generation.

The harness can shape the thing being built

There is a seam here that I think matters more than it first appears.

It would be neat to say that the factory provides machinery while the harness simply provides judgment. The play test rigging of my game shows that the boundary is not purely external.

The rig only works because the game exposes a controlled hook that makes meaningful behavior observable. The project harness required evidence about gameplay, and satisfying that requirement changed the architecture of the thing being built.

That is not judgment layered on top after construction. It is a harness requirement shaping the system itself.

The project harness can therefore impose design obligations necessary to make quality observable. A system may need telemetry because its operational behavior matters. It may need provenance because decisions must be reconstructed. It may need deterministic interfaces because replay is part of the evidence model. It may need test seams or simulation hooks because important behavior cannot otherwise be inspected.

This is a deeper relationship than "build first, evaluate later." The evidence obligations inside the harness can reach backward into design.

Sometimes the ability to prove that a system works has to be built into the system.

The harness is shaped by the build

The influence runs in both directions. The harness can shape the architecture, but building the system also reveals things the harness did not know at the beginning.

That does not mean starting from nothing. A build should enter the factory with substantial craft knowledge, baseline controls, purpose, known constraints, Actors, Articles, and evidence expectations. "Emergent" cannot become an excuse for undisciplined construction. Then reality starts applying pressure.

A reviewer repeatedly catches the same class of defect, which may indicate a missing standard. Two actors regularly disagree, which may expose an unclear responsibility boundary. A fuzzy requirement becomes precise enough to express as an invariant. A judgment that once required a specialist becomes mechanical enough to encode as a gate.

The important thing is not simply that the system learns. It is that the learning has somewhere durable to land.

There are two destinations.

A lesson that is specific to this build belongs in the project harness. A discovery added to domain knowledge changes future reasoning. A discovery converted into a standard changes what future work must preserve. A policy change alters the operating posture. A new evidence requirement changes what future Articles owe.

But some lessons are not really about the project. They are lessons about manufacturing software.

Those belong somewhere else.

The factory needs memory too

Factories should accumulate knowledge about manufacturing itself.

A retrospective, a failed evaluation, or a recurring review finding can expose something that may matter beyond the build where it appeared. Perhaps a decomposition strategy repeatedly created coordination failures. Perhaps a council structure produced redundant review. Perhaps one evidence mechanism caught defects that everything else missed.

Those observations should not immediately become standards.

That boundary matters. Discovering that parallel architectural reviews created disagreement on one project does not mean every future project must use a single architect. It means the factory has learned something worth retaining and making available when a similar situation appears again.

I think of that store as factory memory, or manufacturing knowledge.

The path from observation to mechanism should be deliberate. A lesson begins as an observation. Repeated experience may turn it into a promising pattern. Enough evidence may justify treating it as an established standard. Some standards eventually become enforceable mechanisms.

That ladder is not a second discipline bolted onto the factory. It is the same discipline already used to manage decisions, applied to manufacturing knowledge instead. The object has changed, but the governance idea has not.

Provenance matters here as much as anywhere else. A lesson without context quickly becomes folklore. The system should know where a lesson came from, what evidence supported it, under what conditions it appeared, whether it has been observed elsewhere, and what authority it currently carries. An observation, a promising pattern, an established standard, and an enforced mechanism are not interchangeable.

This gives the learning loop somewhere to persist. A build produces artifacts and evidence. A retro interprets what happened. Project-specific lessons sharpen the harness. Reusable lessons enter factory memory. Later builds can retrieve those lessons when the context matches. Repeated evidence can eventually justify promotion into a standard or into machinery.

Without that store, saying the factory "learns" is mostly metaphor.

With it, learning becomes an architectural capability.

The factory does not simply finish one build with more history behind it. It has a mechanism for carrying earned knowledge into the next one.

The direction matters. I do not want the factory absorbing project semantics just because one experiment taught us something useful. The factory should absorb reusable mechanisms, reusable craft, and reusable manufacturing knowledge. The meaning specific to a project should stay with its harness.

Promotion also has to be conservative. Something working once is not evidence that it belongs in the platform. A retrieved memory should not silently become a standard. A worker cannot remove a difficult constraint because it would be easier to pass without it. A council cannot quietly lower the evidence required for its own verdict. A project should not mutate core factory behavior simply because its local harness finds that behavior inconvenient.

Learning without boundaries quickly becomes self-justification.

There is a test for whether this split is real

The thing I like most about this model is that I can actually be wrong about it.

One of my current experiments is a game, running against the newer factory architecture as a fresh build-out. The factory and target-standard boundary is already in place. The test now is whether that boundary survives as the factory takes on projects with very different semantics.

If the distinction is real, each new project should arrive as a registered target with its own building standards, domain knowledge, Articles, policies, and evidence obligations. The factory should provide the reusable machinery necessary to execute those things without absorbing their meaning.

Nothing in the factory should ever need to learn what a bonk is.

If a new project eventually forces the factory itself to understand game-specific semantics, compliance-specific semantics, or any other domain concept in order to manufacture it correctly, then the boundary is in the wrong place.

That is a much better test than whether the architecture sounds clean on paper.

The next few builds should tell me whether I have separated them correctly.