DeepSeek V4 Pro 0813: Reported Evaluation Gains and the Limits of the Claude Cost Comparison

Written by

in

Evidence note: This article draws on a Hacker News benchmark report, Howard Chen’s published account, deepclaude’s own documentation, and Vision’s internal operator and engineering records supplied to PlayAgit through October 8, 2026. Reported benchmark and pricing claims remain uncorroborated; advertised support is attributed to its publisher. Vision’s measurements describe its own operations, not an independent evaluation of DeepSeek.

DeepSeek V4 Pro 0813’s reported gains make it a candidate for a controlled developer evaluation. They do not establish that it can replace Claude Fable 5 at a small fraction of the cost per completed task. The benchmark comparison, pricing headline and operational records answer different questions.

The reported DeepSWE jump

The supplied Hacker News report puts DeepSeek V4 Pro 0813 at 62.7 on DeepSWE, compared with 12.8 for the preview. Those figures remain uncorroborated.

The supplied finding also describes the DeepSWE website’s title as identifying an August 13 update featuring DeepSeek V4 Pro 0813 and Grok 5.6. That title alone does not verify the scores or establish comparable evaluation conditions.

The useful follow-up is to obtain the underlying evaluation results and settings. A reported improvement within a benchmark is a reason to investigate; it is not a measured improvement in a developer’s own workload.

Three reported comparisons with Claude Fable 5

The supplied finding attributes these comparisons to the same Hacker News report:

Evaluation DeepSeek V4 Pro 0813 Claude Fable 5 Evidence status
Terminal Bench 2.1 87.9 88.0 Reported; not independently corroborated
Cybergym 83.3 83.1 Reported; not independently corroborated
AutomationBench 31.8 29.1 Reported; not independently corroborated

Read literally, the reported scores put DeepSeek just below Fable on Terminal Bench and above it on the other two evaluations. Without established evaluation settings and variability, these comparisons cannot support a general claim of equivalent autonomous coding performance.

Unchanged pricing and the limits of the 5% headline

The Hacker News report says pricing remained unchanged from the preview. That remains an uncorroborated pricing claim.

Howard Chen’s headline describes DeepSeek V4 Pro as costing 5% as much as Claude, an author-reported comparison rather than an independently established task-cost result. The supplied finding identifies no Claude model in that headline. Fresh examination adds an important distinction: the article’s body names Claude Sonnet 4, and the page is dated June 16, 2026. It therefore does not establish a cost comparison between the August 13 DeepSeek version and Claude Fable 5. Chen’s article

For a development team, the relevant denominator is accepted work. A cost comparison needs to include unsuccessful attempts, retries, review and recovery—not merely the price of tokens consumed during one attempt.

Why deepclaude makes the automation question practical

The deepclaude project advertises support for using Claude Code’s autonomous agent loop with DeepSeek V4 Pro, OpenRouter or another Anthropic-compatible backend.

That advertised integration matters because it offers a route to evaluating a different model within a familiar coding workflow. The development question becomes whether the alternative backend can complete the same work under the same permissions, budgets and acceptance criteria.

Advertised compatibility does not establish verified completion or cost savings. Those require execution records from the proposed configuration.

Vision’s operating context: time and tool turns

Vision’s own operator records cover 1,646 runs touching Claude/Fable from August 17 through October 7, 2026, as recorded on October 8. All 1,646 used the Claude CLI. Across 1,646 timed runs, recorded wall-clock duration ranged from 0.0 to 60.5 minutes, with a median of 6.7 minutes; the same sample had a median of 19 tool turns.

These are workflow measurements. The sample is defined by runs touching Claude/Fable, rather than by a controlled assignment of identical tasks to one exact model version. It supplies operating context, not a Fable 5 benchmark.

Time and tool turns are useful evaluation fields because they describe how work unfolds beyond a final answer. A matched comparison would need to retain both alongside its acceptance result and total cost.

Where execution stopped

In Vision’s own 1,646-run Claude/Fable sample from August 17–October 7, 2026, the October 8 record shows 105 runs ending in errors. Recorded categories included 42 resource-budget errors, 11 session-limit errors and 10 runtime errors.

In that same Vision sample and period, the permission gate refused 316 commands across 57 runs.

These categories should remain separate. A refused command records an enforced permission boundary; it does not by itself establish an incorrect model response. Likewise, an error-ending run does not identify whether the underlying cause was model behavior, account capacity, configuration or another part of execution. The supplied aggregate also does not establish how the refusal and error groups overlap.

A backend evaluation should preserve these distinctions so that cheaper inference is not mistaken for a solution to every operational constraint.

What the runs left behind

Vision’s own operator records for 1,646 Claude/Fable runs from August 17–October 7, 2026 show 1,437 runs leaving changes, with 50,999 changed paths in aggregate and 743 commits, as recorded on October 8.

Those figures establish recorded activity. They do not establish verified task completion. A changed path can contain unfinished work, and a commit can preserve an intermediate state. The supplied aggregate does not establish that the path count represents distinct files.

The missing link is task-level acceptance evidence: what was requested, what changed, which checks passed and whether the resulting behavior met the request. PlayAgit’s related coverage of Vision’s run timing, changes and evidence gaps explores the same distinction in another operating context.

The engineering record adds context, not a DeepSeek comparison

Vision’s own engineering log contains 58 measured-and-fixed entries naming Claude/Fable across August 11–October 6, 2026, according to its October 8 record. The three most recent entries were dated October 5, October 5 and October 6.

This record documents engineering attention around the workflow. It does not supply matched DeepSeek runs, comparative repair effort or a measured advantage for either backend.

Its practical contribution is to identify material for a future evaluation: previously encountered problems can become candidate test cases once their starting conditions and acceptance checks are recoverable.

The developer decision: compare accepted outcomes

The evidence supports three separate conclusions: benchmark gains have been reported, deepclaude advertises an integration path, and Vision has recorded substantial Claude/Fable execution activity. None supplies the missing matched cost-and-completion comparison.

A useful follow-up would:

  • Pin the exact DeepSeek and Claude model versions, CLI version and integration configuration.
  • Run the same tasks from identical repository states, with matching permissions and resource budgets.
  • Define acceptance checks before execution and retain their results.
  • Record billed usage, elapsed time, retries, refusals, errors and human recovery work.
  • Compare total expenditure across all attempts against the number of accepted tasks.

This is a proposed evaluation, not a completed experiment. The decision-relevant question remains open: does DeepSeek V4 Pro 0813 lower the cost of accepted work under comparable operating conditions? Neither the reported scores nor the 5% headline answers it.