The Code Became the Cheap Part: Evaluation Ratios, 25KB Agents, and Tiered Routing in the Agentic SDLC

Evidence note: this article draws on three kinds of material, and they are not equal. Most of what follows comes from primary sources — project repositories, vendor engineering blogs, and the personal sites of the practitioners involved — which describe their own work and have not been independently corroborated here; each such claim is marked where it appears. A second layer comes from playagit’s own pipeline records, which are internal measurements rather than independent verification, and are attributed by date and sample size wherever they are used. Nothing below should be read as a benchmark result confirmed by a third party.

Writing the epic stopped being the long pole

The argument that reorders everything else is a claim about which step is now slow. Practitioners building agentic delivery tooling argue that the time to code an epic is no longer the long pole in the software lifecycle, and that the constraint has moved to the rate at which knowledge is produced — faster than a team can absorb it (uncorroborated; this framing appears in the stated motivation of the Distillery project’s own repository and is not independently verified).

If that holds, the interesting engineering problem stops being generation and becomes selection: which output to keep, which to send back, and what record proves the decision was made. Every pattern in this article is a variation on that.

Who is actually building an agentic SDLC, and who is only naming it

The term is crowded. Several public repositories are named "agentic-sdlc" outright and describe themselves as dedicated to building an agentic software development lifecycle — among them one, two, three and a URL-shortener variant built as a demonstration case (claim level: repository self-description only; the scope and maturity of these projects are not independently verified).

What the label is supposed to mean is itself unsettled. The common formulation — that an agentic SDLC positions AI agents as core contributors across building, testing and deploying, with multi-agent systems carrying a project from scratch through to deployment on GitHub Actions, and with orchestration built around dependency graphs, gates, governance and observability — is asserted by the projects themselves and remains uncorroborated. Practitioners also argue that GitHub’s "Continuous AI" terminology and PwC’s published material on the "Agentic SDLC" name the space without supplying an implementable framework; that characterisation is likewise unverified here, and readers should treat it as a contested reading rather than a settled gap.

The assumption CI/CD was built on, and why agent-driven delivery breaks it

The structural claim underneath the whole category is that existing CI/CD tooling was designed on the assumption that humans write the code, and that agent-driven delivery violates it (uncorroborated; asserted by framework authors, not established by independent study). The mechanism proposed is straightforward: pipelines assume a bounded number of changes, each authored by someone who can be asked what they intended. Remove both assumptions and the gate, not the build, becomes the scarce resource.

Whether that is the right diagnosis is open. What can be observed is that gates and governance recur as structural elements across these frameworks rather than optional add-ons — a pattern also traced in an earlier piece on how agentic SDLC frameworks use review roles, delivery gates and shared context.

Six steps, five gates, and the rule that every gate carry real evidence

The most specific published process model in this set belongs to icebox-cli. According to its own process documentation, the framework is gate-driven with six steps, five gates and a production feedback loop, framed as a transition from CI/CD to what it calls AI/CD — Agentic Integration / Continuous Delivery — and eventually AI/AD, Autonomous Delivery. Its documentation further states that the framework is platform-agnostic across GitHub, GitLab, Azure DevOps and Jira with any source-control combination, and that it is open source (claim level: project documentation; independently unverified, including the portability claim, which no third-party report confirms).

The governance rule is the part worth borrowing regardless of the rest. The project’s documentation requires that every gate carry real evidence — actual pull requests, issue comments and gate sign-offs visible in the repository rather than a checkbox in a dashboard (project’s own statement; not independently verified). The failure mode it guards against is familiar from governing agent-driven delivery: a gate with no artefact behind it records that someone approved, not what they saw.

Four patterns that won a startup agent challenge, and why each one is a cost control

Google’s developer blog reports that the strongest multi-agent submissions to the Google for Startups AI Agents Challenge succeeded through foundational software engineering rather than raw model power, and identifies four recurring patterns: bidirectional MCP for inter-agent communication, async event buses for parallel execution, strict unified validation for model fallbacks, and tiered routing to minimise expensive inference calls (claim level: vendor blog post describing its own competition; no independent evaluation of the submissions is available, and the claim that structural practices beat linear prompt chains on resilience, latency and cost is presented there as a conclusion rather than a measured result).

Read as a group, three of the four are budget mechanisms. Tiered routing is explicitly one: send the cheap request to the cheap model. An async event bus converts serial waiting into parallel waiting, which is latency spend. Unified validation across fallbacks means a degraded model’s output is checked by the same contract as the primary’s, so a fallback does not silently become an outage. Only bidirectional MCP is primarily about capability rather than cost.

Fifty thousand lines in just over a week, and where those lines went

Distillery’s repository describes it as an MCP server providing AI coding sessions with persistent, shared team context, and states that it dogfooded itself — capturing its own design decisions and feeding them back into its development (project self-description; not independently verified).

The figure attached to it is the arresting part. The project states it comprises roughly 50,000 lines of Python: about 16,000 production, 25,000 tests and 7,000 documentation, built and shipped in just over a week (claim level: the project’s own repository; these counts are self-reported, unaudited, and the line-count methodology is not stated). Taken at face value, the split is the signal rather than the total: roughly three lines of test and documentation for every two of production code. That is the shape one would expect if generation is cheap and verification is the constraint — but a single self-reported project is not evidence that the ratio generalises.

Why end-to-end agent benchmarks are the wrong bill to pay

The evaluation argument runs against the field’s default. Google’s developer blog on harness engineering argues that end-to-end agent benchmarks such as SWE-bench are expensive, slow, and lack the root-cause diagnostics needed to explain where an agent’s logic broke down, and recommends behavioral evaluations instead: fast, local, unit-style tests that assert on discrete intermediate actions — a specific tool call, a specific file modification — rather than on final string equality (claim level: vendor engineering blog; the cost, speed and diagnostic-gap characterisations of SWE-bench are the vendor’s own assessment and are not independently corroborated here).

The practical payoff claimed is regression safety during change: building inexpensive micro-checks alongside macro benchmarks is said to let a team iterate on system prompts and swap in upgraded models without regression risk (same source; unverified). The underlying logic is checkable by anyone: an assertion on "did the agent call the right tool" fails in a way that names the broken step, while an assertion on the final answer fails in a way that names nothing.

Review moves back onto the developer’s own machine, split into five roles

AgentCheck takes the same instinct into review. Its repository describes it as an open-source local subagent for Claude Code that runs five focused reviewers — logic, security, style, guidelines and product — and states it was built on the premise that hosted pull-request review tools flood pull requests with noise, operate as black boxes, bill per seat, and arrive too late and without access to the developer’s local environment (claim level: project self-description; the characterisation of competing hosted tools is the project’s own and is not independently verified).

Adjacent to the tooling is an argument about what review is for: that in an agentic lifecycle, code review is about trade-offs, knowledge sharing and accountability rather than policing style, naming quirks and trivial regressions (uncorroborated; an argued position, not a measured finding). It is a position with an obvious tension — one of AgentCheck’s five roles is style. The reconciliation implied, though not stated by the project, is that mechanical checks belong to a machine reviewer precisely so human attention can go elsewhere. That division is examined in more detail in an earlier piece on what happens when code is cheap and checking it is the hard part.

Context as infrastructure: the design decisions that used to die in chat

A complaint repeated across these projects is that design decisions and their context are routinely lost — to chat threads and forgotten pull-request descriptions — and that agentic development makes the loss worse by multiplying the decisions (uncorroborated; a shared premise among the tools rather than an independently measured phenomenon). The proposed fix is to treat context as infrastructure: a persistent, shared store that a stateless session can read.

Two shapes of that fix are visible. Distillery implements it as an MCP server, described above. VibrationPlan describes itself as a framework adapting the SDLC for stateless agents through specialised roles, explicit handoffs and living documentation (project self-description; unverified). A separate strand treats context engineering support for agentic coding tools as its own tooling category, implemented for opencode in one such project (uncorroborated).

Worth noting as a pattern rather than a proof: explicit role specialisation shows up on both sides of the lifecycle — as per-reviewer roles inside code review and as handoff roles across delivery. Whether that convergence reflects a real design constraint or simply a borrowed metaphor is open.

Thousands of agents at 25KB each, without containers or orchestrators

The cost argument gets its sharpest statement from the runtime side. Jido presents itself as an Elixir Agent SDK that runs thousands of agents at approximately 25KB each without process-heavy infrastructure such as Docker or Kubernetes (claim level: the project’s own repository; the per-agent memory figure is self-reported with no published benchmark methodology, and no independent measurement is available).

Its stated rationale is unusual: the project argues that existing agentic frameworks were written for humans and encode human work-style assumptions that an LLM must then work around, and that Jido was written specifically for LLMs to code and operate their own agentic flows. It also introduces what it calls a WDLC — a Workflow Design Life Cycle that agents manage themselves, intended to reduce the human intervention needed to build workflows (all project self-description; unverified).

The economics that would make this matter are argued elsewhere. A Stack Overflow blog piece on the economics of agent scale contends that running 50,000 agents in parallel on container and orchestrator infrastructure is costly enough to erode the benefits of agentic programming; the same outlet reports a two-part Leaders of Code conversation with Andi Gutmans, head of Agentic Data Cloud at Google, covering judgment, code review and data activation in the first part and the cost and infrastructure side in the second (claim level: reported by the publication; the cost assertion is an argument, not a published cost study, and neither the figures nor the attribution have been independently confirmed).

What 80 runs of playagit’s own publishing pipeline spent, and on what

Set against all of the above, here is what one small agentic pipeline actually did. In playagit’s own pipeline runs over 80 runs between 2026-08-28 and 2026-09-12, 18,587 topic candidates were discovered across four categories from 213 distinct source hosts (playagit’s own run records, n=80 runs, 2026-08-28–2026-09-12; internal measurement, not independent verification), as of 21:59 UTC on September 12, 2026.

Of what was scored, most did not survive. In the same 80 runs across the same dates, 21,426 scored topics produced 8,065 held before production and 10,079 rejected, with 3,282 going through to production — roughly fifteen percent of everything scored (playagit’s own run records, n=80 runs, 2026-08-28–2026-09-12). Output over that window was 404 posts published, of which 211 are recorded as new and 84 as updated (same source, same window).

The selection ratio, not the throughput, is the number that connects to the rest of this article. Discovery is cheap; the pipeline discovered eighteen thousand candidates. Deciding was where the budget went.

Where the skills for all this are being written down

The teaching material is arriving faster than the consensus. OpenAI is reported to have published a guide for engineering leaders titled "Building an AI-native engineering team," covering the integration of agents into the software lifecycle, which has been translated into Persian (uncorroborated; the existence and contents of the original guide are attested here only by the translation repository). GitHub is reported to offer a certification, "GitHub Certified: Agentic AI Developer (GH-600)," covering building, orchestrating, evaluating and governing AI agents with Copilot and modern lifecycle practices, as catalogued in a community list of agentic AI resources (uncorroborated; the certification’s scope and availability are not confirmed from GitHub’s own materials here).

On the practitioner side, Simon Willison is reported to argue that vibe coding and agentic engineering are converging more closely than he considers desirable, and to maintain a published guide to agentic engineering patterns (uncorroborated as characterisations of his position; readers should consult the sources directly rather than rely on this summary). Separately, one practitioner treats the work of building the platform and codified rules underneath an agentic SDLC as a team topologies problem — an organisational design question rather than a tooling one (uncorroborated; a single practitioner’s argument).

How we apply this in Vision

The figures below are all internal. They come from playagit’s own publishing pipeline and are offered as one operator’s measurements, not as independent evidence about agentic development generally.

The quality gate is the closest local analogue to the evidence-carrying gates described above. In playagit’s own pipeline runs over 80 runs between 2026-08-28 and 2026-09-12, that gate reviewed 507 drafts and returned three outcomes: 358 published, 112 sent back to revise, and 37 rejected outright (playagit’s own run records, n=80 runs, 2026-08-28–2026-09-12). Roughly three in ten drafts did not pass on first presentation — a rejection-plus-revision rate that would be alarming in a human pipeline and is simply the operating point in this one.

The verification ratio is starker, and it is the number that most changed how drafts are written here. Across 2,701 research dossiers holding 10,436 claims, gathered in the same 80 runs between 2026-08-28 and 2026-09-12, 391 claims were independently cross-checked and 1,735 were primary-attested (playagit’s own run records, n=80 runs, same window). That is under four percent independently corroborated and about seventeen percent attested by the subject’s own official source — which is precisely why this article hedges as much as it does, and why the evidence note at the top is not a formality. A pipeline that generates claims far faster than it can corroborate them has only two honest options: publish less, or mark the difference on every line. This one marks it.

The same asymmetry shapes what counts as a gate record rather than a gate result. A decision with no artefact behind it is not auditable after the fact — a failure mode examined in an earlier piece on agent runs that leave no trace. Across these 80 runs the ledger keeps each verdict and its grounds, which is what makes the ratios above quotable at all: they are counts of recorded decisions, not estimates.