Evidence note: This article rests on two kinds of material, kept separate throughout. The first is public reporting and third-party integration work about Anthropic and Claude — news coverage, encyclopedia entries, a company blog post, and open-source repositories. None of it was independently corroborated by this site’s research pipeline, and each such claim is marked where it appears rather than assumed. The second is measurement produced on this site’s own infrastructure: operator run records, an internal engineering ledger, and the research pipeline’s own tallies. Those figures are stated plainly because they were recorded here, with the date range and sample size attached inline — but they describe one deployment on one set of machines, and they are not independent verification of any vendor’s claim.
What Claude Is, and What Anthropic Had Already Added: Citations for Retrieval, Claude 4’s Weapons-Risk Safeguards, Interactive Apps, and Watermarks on Generated Text and Files
The basic description of the product — that Claude is a large language model and AI chatbot developed by Anthropic — comes from an encyclopedia entry and a survey listing of large language models, and remains uncorroborated against primary sources here. The same applies to each of the feature additions below; they are reported, not confirmed.
Ars Technica reported that Anthropic added a Citations API intended to build retrieval-augmented generation directly into its models and reduce confabulation (uncorroborated). CNBC reported that security measures were added to Claude 4 to limit the risk of users developing weapons (uncorroborated). A post on the company’s own blog describes support for interactive apps in Claude; that post exists, but its contents were not cross-checked against any second source, so the capability claim stays open here. CNET reported that Anthropic will add watermarks to text and files generated by Claude — a forward-looking statement, uncorroborated, and treated as an open item rather than a shipped feature; see the earlier piece on watermarks, citations and what a smaller batch of operator runs showed.
None of these four should be read as settled. What follows is a timeline of similarly-sourced entries, and then measurement that does not depend on any of them.
February 2025 to September 2026: Claude 3.7 Sonnet’s Reasoning, Fable 5.1 Reaching AWS in the Week Before 7 September 2026, and the One Entry That Says New Tools Were Suspended Around June 2026
Axios reported on 24 February 2025 on a reasoning-capable Claude 3.7 Sonnet (uncorroborated). Roughly eighteen months later, an AWS weekly roundup dated 7 September 2026 is the basis for the claim that Claude Fable 5.1 became available on AWS during the preceding week (uncorroborated; the roundup is a cloud-provider post, not an Anthropic release note).
Between those two points sits a single, harder-to-place entry: a timeline of artificial intelligence events records Anthropic suspending new AI tools around June 2026. That claim is uncorroborated and unusually thin — one line in an aggregated timeline, with no scope given for what "new tools" covered, how long the suspension ran, or whether it applied to a product surface or an internal program. It is listed here as an open question, not as a fact about the company’s mid-2026 posture. Announcements and their eventual outcomes can diverge sharply, as documented in the case where a patron announcement became a one-time donation.
Built for Long-Running, High-Stakes Work: What Anthropic Claims Fable 5.1 Is For, Across Coding, Scientific Research, and Enterprise Workflows
The positioning attributed to Anthropic is that Fable 5.1 is built for long-running, high-stakes work spanning coding, scientific research, and enterprise workflows. This is uncorroborated: the description reaches this article secondhand, via the AWS roundup covering the model’s arrival on that platform, and no primary announcement was located to confirm the exact framing.
The phrase is worth isolating because "long-running" is the one part of it a deployment can actually test. Duration, interruption, and what a session leaves behind when it stops early are all observable from outside the model. The measured sections below do exactly that — and they measure a command-line tool, not the specific model the positioning describes.
The Surfaces Other People Build On: Model Context Protocol Servers, the API Inside Third-Party Coding Agents, Claude Code Running in Home Assistant, and an Official Directory of Third-Party Agent Skills
The strongest available signal about how widely a model is embedded is not vendor messaging but other people’s code. Several public repositories point that way, and each supports only a narrow claim.
A Model Context Protocol server for video and audio tooling is published on a public server directory. That Anthropic operates an API which third-party developers integrate into their own coding agents is uncorroborated as a general statement, though individual projects are consistent with it: a terminal coding agent, a command-line assistant, a plugin exposing Claude to a general LLM tool, an automated code reviewer, and a dedicated editor front-end. Each shows one integration; together they do not establish the scale of the API business.
Likewise uncorroborated: that Claude Code can be run inside other environments such as Home Assistant, for which the evidence is a home-automation add-on repository — a community add-on, which is not the same thing as vendor-supported deployment. And the claim that Anthropic maintains an official plugin directory listing third-party Agent Skills rests on a repository of project-management skills that presents itself as directory-listed; the directory’s existence and its official status were not verified here and stay open.
One Provider Among Four: Claude’s Native Web Search Backend Beside Google, OpenAI, and xAI, and Claude Behind Multi-Provider Gateways Beside OpenAI, Google, and Mistral
Two integration patterns place Anthropic explicitly alongside competitors, and both claims are uncorroborated.
The first is search: an extension offering native search backends treats Anthropic as one selectable provider next to Google, OpenAI, and xAI. The second is routing: a tool for adding models to a multi-provider gateway lists Anthropic among the destinations beside OpenAI, Google, and Mistral.
The practical consequence for anyone measuring reliability is that in a gateway architecture, the provider that answered a given request is not self-evident from the request itself. That is not a hypothetical concern — it is the reason for a change described further below.
594 Runs on the Claude Command-Line Tool, 17 August to 12 September 2026: 0.0 to 55.1 Minutes, a Median of 5.1 Minutes, a Median of 13 Tool Turns
Vision’s own operator runs that touched Claude — 594 runs recorded between 17 August and 12 September 2026, all 594 of them on the Claude command-line tool — ranged in wall-clock duration from 0.0 to 55.1 minutes, with a median of 5.1 minutes across 594 timed runs, over a median of 13 tool turns (own measurement; not independent verification of any vendor claim).
The shape matters more than the endpoints. A median of about five minutes against a maximum near fifty-five means the distribution is heavily skewed: most work finishes fast, and a small tail runs an order of magnitude longer. A mean would misrepresent this population badly, which is the first reason to distrust any single average offered as an agent-performance number.
What Those Runs Left Behind: 500 Runs with Changes, 6,474 Changed Files, 235 Commits
Of those same 594 runs (Vision’s own records, 17 August to 12 September 2026), 500 left changes behind, touching 6,474 paths in total and producing 235 commits (own measurement).
Three observations follow directly. About 94 runs finished without modifying anything — a meaningful fraction that a pass/fail framing would silently classify as successes. The 235 commits are fewer than half the 500 changing runs, meaning most runs that wrote to disk did not themselves conclude with a commit. And the ratio of changed paths to changing runs works out to roughly thirteen paths per run, which says these were not single-file edits. That last point is why an interrupted run is not a neutral event: a stopped run can still have written. The pattern was examined in detail in an earlier piece on failed runs that had already written to the repository.
What Stopped the Other Runs: 52 Errors, Including 17 Resource-Budget Exhaustions, 11 Session Limits, 6 Turn-Budget Cutoffs, and 126 Commands a Permission Gate Refused Across 39 Runs
Across the same 594 runs (Vision’s own records, 17 August to 12 September 2026), 52 ended in an error (own measurement). The three most common named causes were resource-budget exhaustion, 17 occurrences; a session limit being reached, 11 occurrences; and a turn budget running out after 61 turns, 6 occurrences. Those three account for 34 of the 52; the remaining 18 are not broken out by category in this data, and no claim is made about their composition.
Separately — and this is a different measure, not part of the 52 — 39 runs had a total of 126 commands refused by the permission gate (own measurement, same sample and dates). A refusal is a policy decision by the harness, not a model failure, and folding the two together would inflate an apparent error rate by roughly a third.
The distribution is worth reading carefully. All three named causes are budget or limit exhaustion rather than incorrect output. In this sample, the dominant failure mode was the run being cut off, not the model being wrong — which means the number is at least as much a statement about how the harness was configured as about the model.
After the Correction: Judging a Run from the Tool’s Own Event Stream, and Why a Held Result Was Never a Failed One
Vision’s own engineering ledger holds 33 measured-and-fixed entries naming Claude, dated between 11 August and 9 September 2026 (own measurement). Two of the most recent bear directly on whether the figures above can be trusted.
The first records that the pass/fail judgment for command-line runs was reading error text out of tool output — including stale error strings left over from earlier in a session — and therefore marking runs as failed on the strength of a phrase that no longer described the current state. The fix moved the judgment input to the tool’s own event stream, so a run is judged by the events the tool actually emitted rather than by text that happened to appear in its output.
The second records a scoring defect: a scoreboard comparing providers was counting a held result with the same weight as a failure. A held result is one deliberately withheld pending a gate, not one that broke. The correction removed held results from the applied-rate denominator and narrowed the gate to fire only when something genuinely broke.
Both are counting bugs rather than model bugs, and both ran for some period before being caught. Any reliability figure produced before a correction of this kind is measuring the judge as much as the subject.
Why Every Run Record Since Early September Names Which Provider and Model Did the Work
The most recent of those ledger entries records a deliberately small change: every operator run reply now carries one line naming which provider and which model did the work (Vision’s own engineering ledger, entries dated 11 August to 9 September 2026).
The reason connects back to the gateway pattern described earlier. When requests can be routed across providers, an aggregate reliability number computed over a mixed population attributes to one vendor the behaviour of several. Recording the provider and model per run is the minimum condition for a per-provider figure to mean anything at all — and it is the reason the 594-run count above can be stated as touching one command-line tool rather than an unspecified blend.
Our Research Pipeline’s Own Count: 25 Claude Subjects, 89 Dossiers, 392 Claims, 13 Cross-Checked, and 22 Drafts That Split 15 Published, 4 Returned, 3 Rejected
In PlayAgit’s own research pipeline (38 runs, 28 August to 11 September 2026), 25 distinct subjects naming Claude were researched into 89 dossiers holding 392 claims, of which 13 were cross-checked; the quality gate reviewed 22 drafts and split them 15 published, 4 sent back to revise, and 3 rejected (own measurement).
The cross-check figure is the uncomfortable one, and it is published deliberately: 13 of 392 claims means the overwhelming majority of gathered material about this subject reached the gate with a single source behind it. That is precisely why the first five sections of this article hedge as heavily as they do. It also explains the gate’s output distribution — 7 of 22 drafts did not pass on the first attempt.
How to Read Anyone’s Agent Reliability Numbers, Including Ours
Five questions separate a usable reliability figure from a decorative one.
What population was counted, and over what window? A rate without a denominator and a date range is not a measurement. Is the average the right statistic? Where the median run is 5.1 minutes and the longest is 55.1, a mean would describe no actual run. Were policy refusals folded into the error count? Here, 126 refusals sat next to 52 errors; combining them would have been indefensible. Were held or deferred results counted as failures? That specific defect occurred in this system and had to be corrected. Is a run that produced no changes a success? About 94 of 594 runs in this sample changed nothing, and a binary framing would have counted every one as a win.
The same discipline applies to vendor efficiency and performance claims, which is the approach taken in the review of Vera Rubin NVL72 efficiency claims against agent logs, and in the accounting of timing, changes and evidence gaps across a set of thumbnail and personalization runs. Figures in this article are single-deployment measurements on one set of machines: they are reproducible here, and they are not a general claim about how Claude performs anywhere else.
Implementation Note
Everything in this section comes from measurement recorded on this site’s own systems, with the sample and dates attached.
The run population is Vision’s own operator runs that touched Claude: 594 runs recorded between 17 August and 12 September 2026, all of them on the Claude command-line tool. Across those 594 runs, duration ran from 0.0 to 55.1 minutes with a median of 5.1 minutes over 594 timed runs, at a median of 13 tool turns.
Output from that same 594-run set, same dates: 500 runs left changes behind, 6,474 changed paths in total, and 235 commits. Stoppages from the same set: 52 runs ended in an error — resource-budget exhaustion 17 times, a session limit 11 times, and a turn budget running out after 61 turns 6 times — while separately, 39 runs had 126 commands refused by the permission gate.
The judging changes are recorded in Vision’s own engineering ledger, which holds 33 measured-and-fixed entries naming Claude dated between 11 August and 9 September 2026. Three of them define how the numbers above are produced: run judgment now reads the command-line tool’s own event stream instead of error text sitting in tool output; held results are excluded from the applied-rate denominator rather than weighted as failures; and every operator run reply names the provider and model that did the work.
The publishing side is measured too. In PlayAgit’s own research pipeline across 38 runs between 28 August and 11 September 2026, 25 distinct subjects naming Claude produced 89 dossiers holding 392 claims, 13 of them cross-checked, and the quality gate reviewed 22 drafts — 15 published, 4 sent back to revise, 3 rejected.
Read together, these give one operational rule: the error count and the cross-check count are the two numbers that constrain what can honestly be published. 52 errors in 594 runs describes a harness under budget pressure, not a model producing wrong answers, and 13 cross-checks against 392 claims is why the first half of this article hedges every vendor claim it touches.