Evidence note: This article rests on two kinds of evidence, and they are not equal. The claims about the Stack Overflow blog piece come from a single listing of that post and have not been independently corroborated; they are presented throughout as what the piece is reported to argue, not as confirmed fact. The pipeline figures come from playagit’s own ledger, aggregated over 54 pipeline runs between 2026-08-28 and 2026-09-09, as of 12:14 UTC on September 9, 2026. Those figures are stated as fact because playagit produced and recorded them, but they are self-measurements, not independent verification, and each one carries its source in the sentence where it appears.
What the Stack Overflow blog’s "Dispatches from O’Reilly" piece (2026-08-21) is about
A post titled "Dispatches from O’Reilly: The right amount of spec for agentic development" is listed on the Stack Overflow blog under a 2026-08-21 date. The date, the series framing, and the subject all come from that listing alone. Sources do not independently confirm the publication date, and the description of the piece as part of a "Dispatches from O’Reilly" series about how much specification suits agent-driven software development remains uncorroborated at the time of writing.
What can be said with more confidence is the shape of the question the title poses. "How much spec" is a trade-off question, not a yes-or-no one. Too little specification and an agent fills the gaps with guesses; too much and the specification becomes the code, written twice. The sections that follow describe what the dispatch is reported to argue, then test that framing against a concrete system that has to answer the same question every day: a content pipeline that generates drafts and has to decide which ones are correct enough to publish.
When code becomes cheap, the hard part shifts to deciding what "correct" means
The dispatch is reported to argue that once code becomes cheap to produce, the difficult work moves from writing it to deciding what "correct" means. This characterization has not been independently confirmed, and readers should treat it as a summary of the listing rather than a verified quotation.
The argument is nonetheless worth taking seriously on its own terms, because it matches a pattern that shows up wherever generation outpaces review. If producing a candidate costs almost nothing, the volume of candidates rises until review becomes the bottleneck. At that point the question is no longer "can the system produce this?" but "what standard should the reviewer apply, and is that standard written down anywhere?" A standard that lives only in a reviewer’s head cannot be applied by an agent and cannot be audited afterward. An earlier playagit piece explored this same shift in the argument that verification becomes the job when code gets cheap, with figures from a smaller set of runs than the ones reported below.
The core challenge: a reliable way to check that generated code is correct
The dispatch is also reported to argue that a key challenge of agentic development is building a reliable way to check that generated code is correct. As with the previous section, this is uncorroborated: it reflects how the piece is described, not text that has been independently verified.
If that reading is accurate, the implication is that a specification is only as useful as the check that enforces it. A spec that says "the output must be accurate" is not actionable; a spec that says "every numeric claim must trace to a recorded source, and drafts that fail this go back for revision" is, because a gate can apply it mechanically and a person can inspect the gate’s decisions afterward. That distinction, between a specification as a description and a specification as an enforceable check, is the lens used for the pipeline numbers below.
How much specification is appropriate for agent-driven software development
The listing frames the piece as being about how much specification is appropriate for agentic development, but sources do not confirm what answer, if any, the piece gives. Nothing here should be read as reporting its conclusion.
One way to make the question tractable is to ask where the spec lives rather than how long it is. A specification can sit upstream, in the decision about what to build at all. It can sit alongside production, as the evidence a generated artifact is required to carry. And it can sit downstream, as a gate that accepts, sends back, or rejects the finished artifact. Each location catches a different kind of error, and a system that puts all of its specification in one place tends to pay for it in the others. playagit’s pipeline uses all three, and its ledger records how much work each one does. The related piece on gates, roles, and evidence in the agentic software lifecycle covers the governance side of the same structure.
A specification in practice: playagit’s quality gate over 354 drafts (211 published, 106 sent back, 37 rejected)
The downstream check in playagit’s pipeline is a quality gate that reads each finished draft and issues one of three verdicts. Across playagit’s own pipeline runs (n=54, 2026-08-28 to 2026-09-09), that gate reviewed 354 drafts: 211 were published, 106 were sent back for revision, and 37 were rejected outright. These are playagit’s own recorded counts, not an external audit.
The proportions matter more than the totals. Derived from those same figures, roughly three in five drafts passed on first review, close to three in ten were returned for revision, and about one in ten was rejected. A gate that returns nearly a third of its input is doing real work; a gate that passed everything would be a formality, and one that rejected most drafts would signal that the upstream specification was failing. The sent-back verdict is the most informative of the three, because it is the gate saying the draft is salvageable but does not yet meet a stated standard. Each verdict is written to the ledger as an individual blog-run record keyed to the run’s timestamp and a verdict number, which is what makes the aggregate above traceable rather than remembered.
Evidence as spec: 8,396 claims across 2,456 dossiers, 272 cross-checked and 1,295 primary-attested
The alongside-production specification is the research dossier that each draft has to be built from. In playagit’s own runs (n=54, 2026-08-28 to 2026-09-09), the pipeline held 2,456 research dossiers containing 8,396 individual claims. Of those claims, 272 were independently cross-checked and 1,295 were attested by the subject’s own primary source, according to the same ledger.
The gap between those two numbers is the point. Derived from the ledger figures, well under one claim in twenty had independent corroboration, while roughly one in six rested on a primary source such as a vendor’s own announcement. The remainder, the large majority, had neither and was therefore required to be hedged in any draft that used it. This is what a specification looks like when it is expressed as evidence rather than as prose instructions: the dossier does not tell the writer to be careful, it tells the writer which specific claims may be stated as fact and which may not. The gate in the previous section then checks whether the draft respected that labeling.
Deciding before producing: 19,486 scored topics, 7,300 held, 9,302 rejected, 2,884 sent to production
The upstream specification is the scoring step that decides whether a topic is worth producing at all. In playagit’s own runs (n=54, 2026-08-28 to 2026-09-09), 19,486 topics were scored. Of those, 7,300 were held before production, 9,302 were rejected, and 2,884 went to production, per playagit’s ledger.
Derived from those figures, fewer than one scored topic in six proceeded to production, and nearly half were rejected before any draft existed. This is where the bulk of the pipeline’s filtering happens, and it is also the cheapest place for it to happen, because a rejected topic costs a scoring pass rather than a full dossier, a draft, and a gate review. The held category is worth noting separately: those topics were neither accepted nor discarded, which suggests the scoring specification distinguishes between "not now" and "not ever." The ledger figures do not explain the difference between the 2,884 topics sent to production and the 354 drafts the gate reviewed in the same window; the counts are recorded at different stages and the gap remains an open question that these numbers alone do not settle.
The input funnel: 16,954 candidates from 207 source hosts across 4 categories, ending in 242 published posts
Taken end to end, the pipeline is a funnel. In playagit’s own runs (n=54, 2026-08-28 to 2026-09-09), 16,954 topic candidates were discovered across 4 categories from 207 distinct source hosts. At the far end of the same window, playagit’s ledger records 242 published posts, of which 188 were new and 23 were updates to existing posts.
Derived from those totals, roughly one candidate in seventy became a published post. Two things in the ledger deserve an honest flag. First, the 242 published posts and the 188 new plus 23 updated do not sum: 31 posts are counted as published without falling into either sub-category in the figures available, and the ledger figures used here do not say why. Second, the candidate count of 16,954 is smaller than the 19,486 scored topics in the previous section, which the available figures also do not reconcile; scoring may count re-scored topics more than once, but that is a guess and is not confirmed by the records. Earlier snapshots of this same funnel appear in the pieces on how 7,436 candidates became 75 published posts and what 22 runs revealed about vetting and filtering, and the shape has stayed similar as the sample has grown.
What 54 pipeline runs (2026-08-28 to 2026-09-09) suggest about where the spec should live
Read together, playagit’s own figures from those 54 runs suggest a rough division of labor. The upstream scoring step removed the most volume, rejecting or holding more than sixteen thousand of the 19,486 scored topics. The evidence layer did not remove anything but labeled every one of the 8,396 claims with what could and could not be stated as fact. The downstream gate handled the smallest volume, 354 drafts, but made the most expensive decisions, since each sent-back or rejected verdict discarded work that had already consumed a dossier and a draft.
This is one system’s data, and it is self-measured, so it cannot be generalized to software development at large. But it does bear on the dispatch’s question in one specific way. If the piece argues, as it is reported to, that the hard part becomes defining "correct," then the pipeline figures suggest that "correct" is not one definition but three, applied at three points, and that the cheapest point catches the most. The reason playagit still routes every pull-request-shaped decision through a human, discussed in why pull requests are still gated by hand, follows from the same logic: the last gate is the most expensive place to be wrong.
How we apply this in Vision
Vision is the system that runs playagit’s pipeline, and every number in this section comes from its ledger over the same window: playagit’s own pipeline runs, n=54, 2026-08-28 to 2026-09-09. None of it is independently verified.
- The gate is the spec of record. In those 54 runs the quality gate issued verdicts on 354 drafts, publishing 211, sending back 106, and rejecting 37. Each verdict is a separate ledger record tied to the run timestamp, so a disputed publication can be traced back to the exact decision that let it through.
- Evidence labels are enforced, not advised. Across the 2,456 dossiers and 8,396 claims recorded in the same runs, only 272 claims were cross-checked and 1,295 were primary-attested. A draft that stated any of the remaining claims as settled fact is what the sent-back verdict exists to catch.
- Rejection happens early. Of the 19,486 topics scored in those runs, 9,302 were rejected and 7,300 held before any draft was written, leaving 2,884 to proceed. Vision spends its most expensive review on the smallest set.
- The funnel is measured whole. The 16,954 candidates from 207 source hosts across 4 categories in that window ended in 242 published posts, 188 new and 23 updated. The 31 published posts that fall outside both sub-categories, and the mismatch between candidate and scored-topic counts, are recorded here as open reconciliation items rather than explained away.
What this section does not claim is that these proportions are right. They are what the ledger shows for one site over thirteen days, and the next set of runs will either confirm the shape or change it.