Harness Engineering is a coherent, well-grounded practice for making agent output legible and maintainable at organizational scale. Core insight, encode intent into repository structure so agents infer policy from shape, is sound and validated across four independent personas (CTO, CPO, VPE, Scrum Master) and two specialist lenses (CISO, Scrum Master).
Adoption path is clear: start with domain modeling and encoding nonfunctional requirements into types and lints (3 weeks, immediate payoff), then invest in context routing and maintenance loops as scale demands. Biggest gap is absence of integrated reference implementation that demonstrates all eight ideas running together.
Not ready for naive adoption without architectural discipline; best suited for platform teams already comfortable with strong typing, ownership models, and scheduled observability. Long-term value is preventing agent work from becoming a maintenance tax, by localizing knowledge, making repair mechanical, and keeping proof loops continuous.
ReadyBase score: Good, AI viable with verification. Deterministic, no LLM.
How ReadyBase scores this →Convergent across 4 personas. Transferable, immediate payoff (eliminates instruction debt and compresses context at every job). Adoption cost is upfront (modeling + linting setup ~3 weeks) but compounds as codebase grows. Single biggest lever for harness quality at scale.
Convergent across 4 personas. Transferable, solves context-window ceiling and latency without sacrificing knowledge access. Adoption cost is infrastructure (sidecar, indexing, curation discipline) but enables 10x scaling of agent cognition. Prerequisite for large codebases.
Convergent across 4 personas. Zero implementation cost (discipline only). Isolates harness improvements from worker capability churn, de-risks model upgrades, makes ROI measurable. Qualification cost (~1 week per upgrade) scales sublinearly with harness maturity.
Convergent across 3 personas (CTO, VPE, Scrum Master). Transferable. Prevents drift at scale without roadmap slot competition. Adoption cost ~2 weeks per domain; ongoing curation overhead low relative to prevented rework. Scales to support continuous background work.
Convergent across 3 personas (CTO, CPO, CISO). Transferable, enables human-in-the-loop at scale. Adoption cost is infrastructure (credential broker, audit logging, rollback primitives) ~3 weeks. Tension between audit rigor and latency unresolved but payoff is clear for compliance-sensitive orgs.
Convergent across 3 personas (CTO, CPO, VPE). Transferable, reveals where time actually goes. Low adoption cost (instrumentation) but requires discipline to collect 4 independent signals instead of optimizing single metric. Payoff is targeted investment in feedback loops that matter.
Convergent across 2 personas (CTO, CPO). Transferable. Eliminates handoff latency and reduces context fragmentation. Adoption cost is designing clear outcome contracts and revocation boundaries (~1-2 weeks). Scales by adding parallel trajectories instead of layers.
Convergent across 2 personas (CPO, CISO). Transferable, bridges model generality to org-specific decision-making and reduces secrets-in-weights risk. Adoption cost high (audit of tribal knowledge, schema design, scheduled refresh ~4 weeks) but prerequisite for safe deployment. Security justification is strong.
Flagged by CISO, Scrum Master. Transferable (eliminates configuration drift and stringly-typed ambiguity). Adoption cost is low (centralized format + CI gates) but requires discipline. Payoff is audit legibility and reduced privilege-escalation surface.
Flagged by CISO, Scrum Master. Transferable (eliminates injection vectors at policy boundaries). Adoption cost is upfront (parser validation + strong typing ~2 weeks) but zero incremental cost after pattern is set. Security and maintainability payoff is measurable.
Flagged by CISO, VPE. Transferable (structured error taxonomy + diagnostic instrumentation ~2 weeks). Low ongoing cost, enables agents and humans to remediate without guessing. Payoff is reduced silent failures and faster incident resolution.
Flagged by CISO. Transferable but specialist (migration tax). Adoption cost moderate (adapter layer + fidelity testing). Payoff is reduced latent mistakes in authorization grants and error recovery during tool migrations. Higher value in multi-model deployments.
Inferred by analyst. Domain-specific (targets model-written packages). Adoption cost zero (models enforce automatically). Payoff is non-negotiable quality bar without trade-off. Limited scope (domain packages only) reduces generality.
Transferable (applies to any content-heavy system). Adoption cost low (refactoring existing structure). Payoff is reduced context loss and easier asset management. Limited evidence (one source: blog build) reduces confidence.
Transferable (applies to rendering pipelines). Adoption cost moderate (SSR unification ~2 weeks). Payoff is reduced maintenance surface and consistency. Limited scope (rendering-specific) and evidence reduces generality.
Transferable (applies to all feedback loops). Adoption cost low (discipline + linting). Payoff is preventing repeated mistakes. Evidence is thin (referenced in backlog signal only) and implementation details are vague.
Transferable (applies to all architectures). Adoption cost high (structural refactoring ~4 weeks). Payoff is mechanical enforcement of dependency direction and ownership. Evidence limited to two sources (both code-heavy, specialist context).
Transferable (applies broadly). Adoption cost upfront design (~3 days) then zero incremental cost. Payoff is eliminated stringly-typed mistakes and validation removal downstream. Evidence limited (one specialist source); narrow scope reduces impact.
Transferable (applies to multi-domain systems). Adoption cost moderate (broker design + domain retrofit). Payoff is centralized policy without duplicated knobs. Evidence thin (one mention); scope narrow.
Transferable (applies to legacy modernization). Adoption cost high (interface layer + gradual migration). Payoff is decoupled refactoring without caller churn. Evidence limited (one code source); specialist migration pattern.
Transferable (applies to all codebases). Adoption cost moderate (custom linting + structural checks ~2 weeks). Payoff is legible architectural intent. Evidence limited to 2 personas; overlaps significantly with typed-boundary and encoding-intent ideas, reducing novelty.
Harness Engineering is a practice for shaping agent environments, context, tools, permissions, repository structure, and feedback loops, to improve output from fixed models and coding agents. The repository documents eight core ideas (domain modeling, context routing, fixed-worker qualification, authority separation, maintenance loops, outcome proof, tool legibility, whole-job delegation) through theses, playbooks, and domain examples (Artichoke Ruby state refactoring, blog build, homelab infrastructure, security hardening).
Treats repository structure, types, and linting as part of the harness, agents infer nonfunctional requirements and architectural intent from codebase shape rather than prose. Orthogonal to model choice; transferable across workers. Strongest differentiator: codifies maintenance and proof loops as repository-owned, scheduled work rather than unscheduled drift or roadmap competition.
Repository documents eight mature ideas for scaling agent output, but provides no integrated working implementation or automated qualification procedure for validating harness improvements.
Scales agent cognition: avoids context bloat at 10x load, keeps latency predictable, enables knowledge bases that agents can't fit in a single prompt.
Cost Refactor repository surface into query-able layers (semantic search, navigation contracts, type boundaries); instrument the sidecar that routes; measure hit/miss rates.
Eliminates instruction debt: agents learn the system shape from code, not prose; scales team to 10x without proportional documentation burden; makes architectural repair mechanical.
Cost Audit codebase for loose conventions, codify into types and lints (dependency direction, package topology, trait contracts), retrofit existing code to model, wire lints into CI.
De-risks model churn: each upgrade forces re-proof of the harness (context, tools, prompts) so regressions surface early; isolates model differences from harness quality.
Cost Establish baseline with representative whole-job journeys before any model upgrade; create qualification checklist; maintain per-model instrumentation and A/B instrumentation across upgrade boundaries.
Catches the gap between lab and production at 10x scale: unit tests won't surface resource exhaustion, cascading failures, or integration surprises; real data exposes what matters.
Cost Build ephemeral observability stack per worktree; expose logs and metrics to agent; add production traces to regression suite; instrument deployed outcomes as ground truth, separate from CI.
Prevents drift at 10x scale: scheduled roles catch configuration-vs-code gaps, dependencies creep, and security updates; machines and humans share the loop; fixes don't evaporate.
Cost Define what 'drifting' means per domain (dependency freshness, docs vs config, security advisories); automate discovery; schedule roles to diff and repair; version contracts in code.
Eliminates handoff latency: one agent owns end-to-end, reduces context fragmentation, makes accountability legible; team scales by adding parallel trajectories, not layers.
Cost Define 'consequential boundaries' (approvals, permission gates, undo triggers); give agent closure authority within boundaries; wire sparse delegation tests; instrument trajectory evidence for replay.
Enables human-in-the-loop at scale: agent can experiment in sandboxes (no retry cost), mutations to production are gated and logged, revocable if wrong; audit trail for compliance.
Cost Provision per-worktree sandboxes; implement capability sidecar for narrowly scoped credentials; add mutation ledger; wire up abort and rollback primitives; audit logging to durable store.
Bridges gap between generic model capabilities and org-specific decision-making; enables coherent agent behavior across process changes without retraining or constant prompt updates
Cost Requires audit of tribal knowledge, schema design for domain state, and scheduled refresh loops; initially high lift to externalize, then maintenance overhead shifts from prompt engineering to context curation
Solves context-window ceiling and latency; agents make better decisions with less noise by selecting relevant slice on-demand vs. dumping everything upfront
Cost Needs sidecar/broker infrastructure, navigation contracts, and indexing discipline; pays back through reduced context cost and faster inner loops once operational
Eliminates handoff overhead and context loss; agent owns outcome coherence instead of humans orchestrating fragmented steps; reduces synchronous human attention demand
Cost Requires sparse, durable intent documentation and trust in agent autonomy; upfront cost is designing clear outcome contracts and revocation boundaries; ongoing cost is human judgment at the right moments only
Shifts authority from prose guidelines to mechanical enforcement; agent learns org standards from reading examples, reducing instruction demand and drift
Cost Requires architectural discipline, custom lint, and structured package topology; one-time cost to refactor and document intent; ongoing cost is keeping structure coherent as it ages
Enables human-in-the-loop work with consequence recovery; agent can explore safely without delegating all keys; revocation is clean boundary between environment and outcome
Cost Needs credential broker, endpoint binding, read-only agent identities; infrastructure cost upfront; ongoing cost is audit trail management and policy review
Makes harness investments cumulative and predictable; avoids chasing moving targets; identifies which improvements are env vs. worker-driven
Cost Discipline to hold steady during urge to upgrade; cost is accepting temporary capability gaps; benefit is clear ROI measurement and durable leverage on known worker
Reveals where time actually goes (idle loops vs. slow verification vs. human bottlenecks); enables targeted investment in feedback loops that matter
Cost Requires instrumentation and honest telemetry; low cost to add timing; ongoing cost is interpreting signals and avoiding false correlations
Reduces onboarding friction and ensures consistency across new code without tribal knowledge or per-PR review cycles
Cost Requires upfront investment in domain modeling, typed boundaries, and linting rules; payoff compounds as codebase grows
Cuts agent context waste, improves decision velocity, and makes maintenance loops scale without exponential documentation bloat
Cost Needs curator discipline and versioned authority contracts; infrastructure to index and route selectively
Isolates variables, makes harness improvements measurable and reproducible, prevents chasing tool changes without learning their impact
Cost Discipline to hold steady despite allure of new models; qualification work scales with agent capability jumps
Shifts maintenance from roadmap slot competition to continuous background work; keeps knowledge current without manual coordination
Cost Requires scheduled observability roles and runbook infrastructure; ongoing curator effort to compare docs against config
Makes architectural intent legible to agents and reviewers; catches drift at commit time rather than production incident
Cost Upfront structural refactoring and custom lint rules; maintenance burden on architecture stability
Reveals where time actually goes; prevents optimizing inner loop while blocking on external waits or human approval gates
Cost Instrumentation work; discipline to collect and interpret four independent signals instead of one metric
Externalizing sensitive process data reduces the risk of embedding secrets, PII, or proprietary information in model weights or prompts where access control is implicit and audit trails are weak.
Cost Requires inventory of all organizational data sources, access-control retrofit, and audit instrumentation at retrieval boundaries.
Decoupling capability from authority enables least-privilege mutation patterns; audit trails become mechanical and revocation is trivial, unlike post-hoc access reviews or secret rotation.
Cost Requires credential broker infrastructure, audit logging on every boundary crossing, and policy enforcement at tool invocation time.
Eliminating stringly-typed comparisons removes injection vectors at policy boundaries; semantic values carry validation evidence that can be audited and compared precisely.
Cost Requires strong typing discipline and parser validation test coverage; modest up-front cost yields audit legibility.
Single source of truth eliminates configuration drift and conflicting policy specs that create exploitable ambiguity during privilege escalation or secret-handling reviews.
Cost Requires centralized manifest format, validation at every ingest point, and CI gates rejecting inconsistencies.
Legible violation signals allow agents and humans to remediate without guessing; silent failures hide security-relevant invariant breaks until post-incident review.
Cost Requires structured error taxonomy and diagnostic instrumentation; low cost once architecture contracts are formalized.
Migration risk compounds when tool invocation patterns change; preserving model-learned action names and error handling patterns reduces latent mistakes in authorization grants and error recovery.
Cost Requires adapter layer and complete fidelity testing on new host; moderate cost if original tooling is well-specified.
Turns maintenance work that drifts unscheduled into infrastructure that runs mechanically; agents can keep work moving without waiting for roadmap slots
Cost Requires repository contract ownership and scheduled domain validation roles; initial lift ~2 weeks per domain
Removes instruction demand and compresses context; agents learn architecture from shape, not prose; consistency is recoverable
Cost Upfront modeling cost (~3 weeks); pays back immediately in reduced prompt overhead per job
Makes failure actionable by isolating whether a miss is worker capability or harness gap; prevents chasing model updates without measurable effect
Cost Discipline only; requires qualifying whole journeys (~1 week per worker upgrade); no implementation cost
Prevents accidental prod dependencies on dev helpers; makes package boundaries legible to agents; simplifies testing and shipping
Cost Linting rules + structural checks; ~1 week to retrofit existing repo
Removes stringly-typed mistakes; machine knows context without caller rebuilding naming policy; eliminates repeated validation downstream
Cost Upfront design (~3 days); zero incremental cost after pattern is set; agents enforce 100% coverage automatically
Keeps prompt lean while maintaining access to durable knowledge; agents select only what's needed for the next decision
Cost Requires context sidecar or materialized index; ~2 weeks setup for initial catalog
Separates user-facing correctness from internal consistency; catches drift that tests miss; builds confidence in production behavior
Cost Requires instrumentation and trace access; ~1 week to expose per critical path
Eight convergent theses well-reasoned and transferable (domain modeling, context routing, fixed-worker qualification, authority separation, maintenance loops, proof at outcome boundary, tool legibility, whole-job delegation). Clear separation of concerns across docs/, playbooks/, sources/, evals/. However, architecture itself is distributed, no monolithic codebase or reference harness demonstrating ideas running together. Strong conceptual coherence masks incomplete realized structure.
Theses are mature; playbooks provide procedural scaffolds; domain examples (Artichoke state refactor, blog build, homelab) are real and complex. But no monolithic working implementation. CI/CD integration incomplete (ReadyBase: 0/10 build, 0/10 CI/CD). Qualification procedure for fixed workers outlined but not automated. No reference system demonstrating all eight ideas operational. Documentation-to-shipped-code ratio is 10:1. Not production-ready as-is; best case is mature guidance awaiting implementation.
CISO findings show six concrete security mitigations (externalize process data, separate capability from authority, typed boundaries, canonical manifests, diagnostic legibility, semantic-preserving migrations). Credential custody via sidecar, read-only agent identities, endpoint binding, audit logging all present in design. Threat model not formalized and no security review evidence in repo. Risk: externalizing sensitive data without access-control retrofit remains high. Revocation boundaries clear in theory but no production audit trail examples.
Eight core ideas are highly transferable across domains; guidance spans infrastructure, platform teams, security, operations. Domain modeling and context routing apply to any agent-heavy codebase. However, reuse requires significant local adaptation: no vendorable library, no off-the-shelf harness components, no API for shared maintenance loops. Playbooks are scaffolds, not templates. Examples (Artichoke, blog, homelab) are domain-specific; pattern extraction tax on adopter is high. Theses are reusable; implementations are not.
Repository is thorough (40+ markdown files, theses, playbooks, domain modeling examples, feedback loops, authority patterns). AGENTS.md, playbooks/README.md, docs/README.md all present. However, ReadyBase: 15/15 (README 3 days old). Documentation overstates integration and automation: authority separation, maintenance loops, and qualification procedures are designed but not implemented. Playbooks read as design documents rather than runbooks. No getting-started guide for practitioners. No video walk-through or worked example showing all eight ideas together.
ReadyBase: 0/3 test quality, 0/10 CI. No automated tests for theses or playbooks. Evals/ directory exists but evaluation framework is procedural (checklists, before-after guidance, representative journeys) not executable. No regression tests for harness changes. No qualification checklist automation. Domain examples (Artichoke, blog, homelab) have tests but are external to this repository. Testing is a claimed capability (proof at outcome boundary) but absent from repo itself.
Repo exists to demonstrate patterns; making intent legible through structure is the core deliverable. Direct payoff for extracting reusable patterns.
Understanding repo architecture requires indexing all patterns; this pattern itself demonstrates context scoping. Recursive payoff.
Critical for extracting patterns reliably; fixes model behavior so pattern catalog doesn't churn with worker upgrades. Prerequisite for stable inference.
Repo patterns must stay current; this pattern operationalizes that. Enables continuous pattern validation without roadmap competition.
Orthogonal to pattern extraction; safety infrastructure for agent mutation, not understanding codebase. Different trust boundary.
Extracting patterns is outcome; reframe clocks as: agent inference quality, time-to-catalog, pattern recall accuracy, adoption friction. Apply narrowly.
Extraction is intrinsically collaborative (human validates patterns); long single trajectory breaks at validation gates. Wrong decomposition.
Repo contains implicit practices; externalizing them into docs + executable specs is exactly the extraction task. Limited scope (this repo only).
Pattern catalog must be canonical; this ensures catalog stays consistent as repo evolves. Non-negotiable for trustworthy extraction.
Pattern relationships (which patterns block/enable which) need structured reasoning. Prevents stringly-typed pattern conflicts.
When pattern extraction fails, agent needs to know which invariant broke (coverage, circularity, missing source). Improves convergence.
Migration is admin work, not pattern extraction. Applies if worker changes mid-project; deferred to epoch requalification.
Applies to generated code; pattern extraction is analytical, not generative. Different concern.
Applies to content structure; repo patterns are already dispersed (docs, code, lints). Wrong modality.
Rendering-specific. Pattern extraction is about understanding, not serving content.
Pattern catalog itself becomes infrastructure; use reviews/feedback to identify which patterns are stable vs. emergent. Feed back into loop.
Key pattern from repo; make dependency structure legible so extraction reveals ownership boundaries and makes intent mechanical.
Pattern itself uses immutable types; codify as reusable pattern with examples from repo. Demonstrates applicability.
Broker is infrastructure, not pattern. Extracting broker design is separate from understanding repo.
Migration pattern; extraction is static analysis, not refactoring. Orthogonal concern.
Pattern catalog IS executable prompt; use repo lints + tests to validate extracted patterns so extraction feeds back into codebase.
Start with adopted patterns in order: (1) encode intent into repo structure via canonical types + lint rules (2) build semantic index + routing for whole-repo context (3) define pattern domain model + invariants (4) lock agent + baseline (5) wire extracted patterns back into CI as lints. Run in sequence; pattern extraction is self-proving (each enables next). Biggest risk: extracted patterns conflict with existing code culture; mitigate with early validation loop (review extracted patterns, test on small subsystem before scaling indexing).