Is Dual-Wielding Worth It?Two Blades, One Hand

Dual-wielding means having a GPT model check a Claude model's work, and the other way round. Does the second vendor add anything a second run of your own model would not? On one task, reviewing experiment designs for flaws, mostly no: a second review was worth having whoever wrote it, and what mattered was the tier of model, because a cheap review found far fewer of the defects that decide a result.

The name comes from a section of readers' notes headed "Dual Wielding" on Zvi Mowshowitz's AI blog, Don't Worry About the Vase, and by July Zvi himself was saying "ask both, and compare." That made Evan curious, because the advice skips a question: would a second review from the same vendor do just as well? Appendix A8 has the history.

The Experiment

The task is adversarial review: reading a design to find what is wrong with it. Evan runs a benchmark on what AI coding agents cost, and each experiment starts as a design package: the plan, the scripts that run it, and the code that analyses the results. Before anything runs, a reviewer model is asked what in the package would invalidate the experiment or blow the budget, and it answers with a list of flags, one for each problem it suspects.

Ten models from the two vendors reviewed three designs. A configuration is one model at one effort setting (the vendor's dial for how long the model thinks) with one brief, the written instructions for the review. Nineteen configurations were run, ten of them two or three times, making thirty-eight runs.

Dual-wielding is the second or third row; the first is the control.

How the Defects Were Counted

Every flag from every run went into one pool, and flags describing the same problem were merged, so two reviewers who saw it get the same credit. An AI judge, Opus 5, ruled on each merged defect, and only those it ruled real are counted: 320 confirmed defects, 71 of them on an answer key, a list of known defects drawn from a few early reviews. Confirmed means accepted by that judge, not proven true; appendix A1 has the method and three checks on it. Each was also rated by consequence, in three tiers.

What a Second Review Adds

Count a second review as a buyer would: what the second list adds that the first did not have, over every pair of reviews read both ways round.

The vendor question is the bottom two rows: a different configuration from your own vendor added as many new defects as the other vendor, tier A included, and the share that is new differs by about three points. The rerun adds fewer partly because it repeats more of its own first list, and partly because most of the configurations that were rerun were cheap ones with short lists.

Different Vendor, Similar Overlap

Overlap tells the same story. Take two reviews and ask: of all the confirmed defects either one found, what share did both find?

Reviews from different vendors overlap a little less than reviews from the same vendor, 0.34 against 0.39, and part of that gap is list length: the newest GPT models at their medium setting write short lists, which score low against long ones. Among pairs of similar length the gap is three points. Only running the identical configuration again lifts overlap, and only to about half; appendix A3 explains why even that lift is an upper bound.

The Defects That Matter

Counting every defect once treats a stale comment like a wrong denominator in the metric. So one coder from each vendor, Fable 5.1 and GPT-6 Astra, rated all 320 by consequence under a rubric fixed in advance, A meaning the defect would decide the result or the budget call, B that it would degrade the result, C that the experiment is unaffected, blind to who had found each; they agreed on 67 percent, and Opus 5.5 settled the 74 still disputed by reading the code. Evan's own labels on twelve, made before he saw any model's, matched the final tiers on only four and were mostly graver, so by the rule fixed in advance the tiers are reported as the models' judgement (appendix A9).

The tier of model decided how many of the 50 tier-A defects a review found: 20 to 27 for one strong review, GPT-6 Astra or Fable 5.1 at high or Opus 5.5 at xhigh, and 3 to 17 for one costing about three cents a defect or less, which also found fewer at every other tier. The strong runs' extra finds were 14 to 29 percent tier A, near the pool's own 16 percent: a cheap review is not skipping trivia, it misses everything in proportion.

The vendor still did not matter. On tier-A defects, reviews from the same vendor overlapped at a median 0.44 and reviews from different vendors at 0.39; among pairs of similar length, both at 0.50.

Checking Each Other's Work

The other use of a second vendor is as a checker: hand it your model's flags and ask which are real. To test that, the 134 flags from one Opus 5 review went to three verifiers: two Opus 5 runs with different briefs, and GPT-6 Astra.

They agreed on the clear cases and split on the borderline ones. Swapping in the other vendor changed about as many verdicts as swapping the brief, 25 flags against 20. Astra said "cannot tell" far more often than either Opus 5 verifier, and its caution cut both ways: it refuted one of six real defects planted among the flags, where the first Opus 5 verifier passed all six. This is agreement, not correctness: nobody has audited which side of a disagreement was right.

The One Difference: Cost Accounting

Sorted into eight kinds and a catch-all by coders not told who raised each defect, the vendors' finds differ by more than chance in only one kind, once the test is corrected for looking at every kind at once: cost and meter accounting, about five times as large a share of Claude reviewers' finds as of GPT reviewers'. The designs are cost benchmarks and the Claude side was larger, so this describes the data rather than proving a trait; nor can it separate the models from Claude Code sessions reading the spend files as easily as the code (appendix A6).

What to Do

Zvi's case, in May, was that "two different minds of similar strength are often better than one." On this task the strength did more of the work than the difference: a second mind was worth having from either vendor, and your own model, run again or given a different brief, was nearly as good as the other vendor's, with no second subscription and no second toolchain.

So buy coverage first. Per confirmed defect the cheap configurations win: three cents a defect for GPT-5.6 Terra, GPT-6 Sol and Opus 5.5 at low, against 23 cents for Fable 5.1 at high, and per tier-A defect the order barely changes. But each of those reviews finds far fewer of the defects that matter (Figure 7, right), and the answer key, which needs no severity rating, agrees: the cheap GPT configurations found 16 to 37 of its 71 rows per run and Opus 5.5 at low 45 to 49, the strong ones 56 to 63. Buy one strong review, then a second from whichever reviewer is cheap.

The case this data cannot close is a shared blind spot: two runs of one model can miss the same thing for the same reason. A second vendor is one hedge against that; so is a different brief, a test suite, or a human reader. Nothing here showed such a defect, which is evidence against it on this task and silence about every other.

Everything here is one task, adversarial review of three experiment designs from one repository, judged by one vendor's model, with twenty-six rerun pairs from ten configurations standing in for run-to-run noise. What was measured is which confirmed defects each review found and what verifiers and judges made of them; not whether the defects got fixed, whether the experiments' decisions changed, or what it costs a person to reconcile two reviews by hand. Writing, factual research and ordinary coding were not tested. The severity tiers are a model-assigned judgement of consequence, not an observed one. Details, data and scripts: github.com/edd426/dual-wielding-data.

Appendix: Method, Checks and Background

The method, the checks and the extra views behind the main article, for readers who want them.

A1. How the Count Was Built and Checked

The three designs test whether sending hard work to the strongest model is cheapest per correct answer, whether splitting a task across models saves money, and a procedure that runs a cheap sweep and then a strong judge. The task is open-ended, which an overlap study needs: even a design that has already been reviewed yields hundreds of distinct objections.

The aim was a fair count of what each review found, not a list of every defect in the designs. Flags describing the same problem in different words were merged into the catalogue, the running list of merged defects that each new run's flags were matched against, and the Opus 5 judge ruled on each merged defect in two passes.

Of the 320 confirmed defects, seventy-one are on the answer key: 48 written in August from one GPT-5.6 Sol review and one Opus 5 review, and 23 added in September that other reviews had found and those two had missed. The other 249 came out of the audit, the judge's ruling on every merged flag not already on the key. A screen for duplicates at the same location could bring the total as low as 282; a defect nobody raised is in neither count.

Checking every flag cost more than raising them. From 10 to 19 September the Claude side came to about $830 at API list price, more than half of it on grading and auditing, and the fifteen runs of 24 September added about $240; the GPT runs were metered in Codex subscription points, which have no dollar price. Nobody was billed either: both vendors' runs were on subscriptions. That cost is why there were only three designs.

The count could favour one vendor through whose reviews wrote the answer key, whose model judged the flags, or whose reviewers wrote the most. Each was checked, and none moved the result.

The answer key. A key written from one review inherits that review's misses, and a ranking on it follows its author. On the original 48-row key, written from one GPT-5.6 Sol review and one Opus 5 review, Sol ranked first, and fell to third once 23 defects neither source review had caught were added. That was not a vendor effect: as a share of everything a configuration found on the key, those 23 were a fifth to two fifths for every configuration of either vendor, and the other Sol configurations sat in the middle of that range. In panel A the key's GPT-5.6 Sol author scored 4 under a stricter grading rule, about 8 to 12 on the scale the others were graded on; the misses were not one vendor's blind spot. Those 23 were selected as misses, so they do not measure what a second review adds in general; the table in the main article does.

The judge. Every verdict in the pool came from Opus 5, so a sample of 80 defects, 60 it had accepted and 20 it had rejected, with both vendors' flags represented, was re-judged by GPT-5.6 Sol at high effort and by a fresh Opus 5 run, with planted defects mixed in. Both caught every planted defect, and neither leaned toward either vendor's flags. In panel B the three judges agree on 85 to 91 percent of real-or-not calls; the two re-judges matched each other on 91 percent. The original verdicts came from two passes and the re-judges from one, so the 85 percent mixes that difference with ordinary rerun noise, and judging in one pass rather than two moved verdicts at least as much as the judge's vendor. Confirmed by a judge is still not the same as true: nobody has settled the disagreements.

Volume. A reviewer that raises more flags gets a bigger share of the pool, and effort settings are not comparable across vendors, so every comparison is read with that in mind. In panel C, volume, not vendor, decides a run's share.

A2. Every Pair on One Grid

A3. Where a Rerun Beats a Stranger

On the 71 answer-key defects the medians are 0.64 against 0.54, a gap that is list length: among pairs of similar length it is 0.64 against 0.66. On the 249 found beyond the key they are 0.15 against 0.05, and on the whole pool 0.53 against 0.37.

So on the answer key a rerun and a pair of different configurations are indistinguishable once list length is matched; the raw gap comes from the newest GPT configurations, whose short lists score low against everyone. The whole rerun advantage is beyond the key, where a rerun re-finds about three times the share a stranger does, on sets so small that a typical pair shares two or three items in twenty. Two features of the method favour the rerun there: the catalogue grew within each configuration's run sequence, and matching across configurations was done in order. Treat the gap as an upper bound.

Counting partial matches as hits, where a reviewer named the right place but not the mechanism, all twenty-six rerun pairs overlap near 0.91 on the answer key, from 0.73 to 0.99, against 0.88 for pairs of different configurations. The strict numbers depend on how precisely a reviewer has to describe a defect.

A4. Temperament, in Three Panels

What differs is temperament, and only part of it is the vendor's. Volume is a setting: at high effort Fable 5.1 raised about twice as many flags beyond the answer key as GPT-6 Astra, with the same share confirmed and about ten times the output tokens (panel A), but effort levels are each vendor's own, and Claude Code ran many turns with tools where Codex ran one. The same pair, ringed in Figure 3, overlaps at 0.48: Fable found about twice as many defects that Astra missed as the other way round, but it also raised nearly twice as many flags, so part of that gap is Fable writing more down, not seeing differently.

Abstention looks more like the vendor: in panel B, GPT-6 Astra said "cannot tell" where the Opus 5 verifiers committed. And which model you start with matters more than which vendor. Panel C runs from GPT-6 Astra at 0.65 and GPT-6 Sol at 0.55 to 0.66 down to Sonnet 5 at 0.19 to 0.41: GPT-6 Sol and GPT-6 Astra repeated themselves most and Sonnet 5 least, partly because of how precisely the grader, the model that scored each review against the answer key, required a defect to be described.

A5. Every Comparison Side by Side

The rows are overlap of confirmed defects over every pair and over pairs of similar length, overlap on the answer key alone among pairs of similar length, verifier disagreements over the same 134 flags, and judge agreement with the original verdicts. Similar length means the two lists are within ten confirmed defects of each other. Among pairs of similar length the vendor gap is three points on the whole pool and on the answer key alike. The overlap rows reuse the same 37 reviews; the verifier and judge rows are the separate checks described in the main article.

A6. Cost Accounting, in Detail

Both counts here ran on an earlier, 287-defect version of the pool (see the method notes). The count by vendor came first. Among the defects found beyond the answer key, Claude had twice as many configurations raising them, and defects raised by one vendor only ran just above what that predicts. No vendor owns a class of defect. The strongest single result: four defects were raised by five or more Claude configurations and by no GPT one, against about one expected by chance.

The count by kind came second. Opus 5 sorted the claims blind, without being told which vendor raised each, into eight kinds and a catch-all, and GPT-5.6 Sol coded the same 287 claims blind as a second coder. Only one kind differs between the vendors by more than chance once the test is corrected for looking at every kind at once: cost and meter accounting, 3 percent of GPT finds against 15 percent of Claude finds, with 27 such defects raised only by Claude reviewers, none only by GPT and 2 by both. It includes two of the four defects above. The two coders agreed on 85 percent of labels and put the gap at the same size.

Several other kinds differ two-fold between the vendors in Figure 6 but do not survive the correction. The designs are cost benchmarks and the Claude side was larger, so the one gap that does survive describes the data rather than proving a trait. It is still the only difference by kind in the data.

A7. The Newer Models

Three models shipped while this was being written, GPT-6 Sol, GPT-6 Luna and Opus 5.5, so each was run three times per configuration, Opus 5.5 at both low and xhigh, on the same brief and graded and audited the same way, with GPT-5.6 Luna run three times as the comparison for GPT-6 Luna. These are the fifteen runs of 24 September. Key defects are counted out of the 71 on the answer key; overlap is on the whole pool.

Opus 5.5 at xhigh set the highest scores in the study and was the only one to add much to the pool: 22 confirmed defects, more than any configuration before it; Opus 5.5 at low added two and the three GPT configurations none. Each new model overlapped the model it replaced about as much as that model overlapped itself when rerun, so along a vendor's own upgrade path the new model reads like another rerun of the old one. Effort labels are each vendor's own, so medium on GPT-6 need not buy what it bought on GPT-5.6.

A8. Where the Idea Comes From

Readers of Don't Worry About the Vase, Zvi Mowshowitz's blog about AI, were describing the practice by October 2025. By March 2026 his posts carried a section of readers' notes headed "Dual Wielding", which gives this post its name; they were mostly about the friction of running both tools. In May he said it himself: "two different minds of similar strength are often better than one." By July, in a post on GPT-5.6 Sol, it was an instruction: "ask both, and compare."

A9. How the Defects Were Rated

The rating was designed, and its analysis and decision rule fixed, before any label existed, with a dated log of every change after launch. Each of the 320 confirmed defects was shown as text alone, the answer-key row or the merged defect's description, with nothing naming the reviewer, vendor, run or date. The rubric asked about consequence for the experiment, not effort to fix or how the flag was worded. Tier A: unfixed, the conclusion or the budget decision would be wrong or the run would not complete, as with a wrong denominator in the metric, a leak that lets the tested agent see the answer, or a cost meter that misses a billed component. Tier B: the conclusion stands but is noisier, biased or dearer, as with an unrecorded seed; tier C: nothing changes, as with a stale comment.

Two rules of thumb settled close calls: tier A if Evan would have to rerun the experiment after the fix, B if he could re-analyse the data he has; B if any number in the report changes, C if none does. A real but misdescribed defect was rated on what is actually wrong, and the coders were told to put a claim too vague to rate in tier B and say so.

Coders and planted items. The coders were the strongest model on each side, Fable 5.1 at high through Claude Code and GPT-6 Astra at high through Codex, each labelling every defect in the same shuffled order without seeing the other's labels. Six planted items, two per tier with the tier fixed in advance, were mixed in and kept out of every statistic; both coders labelled all six correctly.

Agreement and adjudication. The coders agreed on 67 percent of the 320, a Cohen's kappa of 0.40. GPT-6 Astra was the strict one, with 132 in tier A against Fable 5.1's 51. In one round of reconciliation each saw the other's label and reason on the 106 disputed defects and could change its mind once; Astra moved on 30, Fable 5.1 on 6. Opus 5.5 at xhigh adjudicated the 74 still disputed with read access to the design packages, the coders' names hidden, and put 69 in tier B, 5 in C and none in A, so every tier-A defect is one both coders placed there. The final count is 50 in tier A, 234 in B and 36 in C.

Loyalty. Both coders were also reviewers in the study, so the share of tier A was compared among defects found only by Claude reviewers, only by GPT reviewers, and by both. Fable 5.1 gave 0.10, 0.21 and 0.23; GPT-6 Astra, stricter throughout, 0.27, 0.46 and 0.59. Astra rated the GPT-only finds graver than the Claude-only ones by 19 points, but Fable 5.1 leaned the same way by 11 points, against its own vendor, so the flag fixed in advance, for an own-vendor lean in one coder that the other does not share, was not raised. Only 24 defects were found by GPT reviewers alone, so that share is coarse. The final tiers give 0.09, 0.21 and 0.23, close to Fable 5.1's; of the 50 tier-A defects, 30 were found by both vendors' reviewers, 15 by Claude reviewers only and 5 by GPT reviewers only.

The anchor. Evan's labels were the only human ones. Twelve defects drawn at random (cut from 25 at his request before any label existed) went to him as text, and he labelled them before any model's label was read. The final tiers matched him on four, Fable 5.1's first pass on four and GPT-6 Astra's on five. He was graver on most of the rest: the four defects he put in tier A are all tier B in the final tiers. He also noted that some items gave him too little context to be sure. Twelve labels cannot say where the line should be, but they point the same way as the adjudicator's putting no disputed defect in tier A: if anything, the final tier A is drawn tightly.

The decision rule. If the configurations cheapest per defect were also cheapest per tier-A defect and found at least half the tier-A defects the dearest found, the advice would stand with a caveat; if they found under half, or if the strong runs' surplus was at least half tier A, it would change to coverage first. The vendor conclusion would change only if the tier-A overlap gap between same-vendor and other-vendor pairs of similar length exceeded 0.10. And if the coders disagreed with Evan on five or more of his twelve, the tiers would be reported as model-assigned and no advice would change on their strength alone.

How it resolved. The coverage clause fired: GPT-5.6 Terra found 9 or 10 and GPT-6 Sol 8 or 9 where the dearest found 23 to 27, and Opus 5.5 at low, at 11 to 17, was the one cheap configuration near half. The surplus clause did not, at 14 to 29 percent tier A against 16 percent in the pool. The vendor gap was 0.05 over all pairs and 0.00 among pairs of similar length, so the vendor conclusion stands. The anchor clause fired, so the tiers are reported as model-assigned.

The advice changed anyway, on evidence that does not rest on where the tier-A line falls: the cheap runs found fewer defects at every tier, the strong runs' surplus sat at the pool's own tier-A rate, and on the answer key, with no rating at all, the cheap GPT configurations found 16 to 37 of the 71 rows per run, Opus 5.5 at low 45 to 49, and the strong ones 56 to 63. That step was a judgement made after Evan's labels came in, and the log records it as one. The rating cost under $15 at list price on the Claude side; the GPT-6 Astra passes were metered in Codex points.

Per tier-A defect the configurations rank almost exactly as they do per confirmed defect; what separates them is coverage. Each configuration's runs:

The surplus, pair by pair: the first Opus 5.5 xhigh run found 96 defects that the first GPT-5.6 Terra run did not, 17 of them tier A; the first GPT-6 Astra run found 42 that the first GPT-6 Sol run did not, 12 of them tier A. On the answer key, 18 of the 71 rows are tier A, against 32 of the 249 found beyond it: the key is not a list of the gravest defects.

A10. Method Notes

• Confirmed. The pool counts a defect only on a "real" verdict. The temperament panel's confirmed shares, 81 and 84 percent, also count "real but misdescribed", as that arm's own scorer did; on the pool's rule they are 64 and 68.
• Matching order. Flags were matched into the catalogue in the order the runs arrived, so later runs met a fuller catalogue and a configuration's reruns met their own earlier clusters of merged flags. Nothing was rematched once the catalogue was complete. This favours reruns beyond the key, which is why A3 calls that gap an upper bound.
• Pairs are not trials. The 666 pairs are combinations of 37 runs, the 26 rerun pairs come from ten configurations, and every complete run read the same three designs from one repository. No confidence interval would survive that, so none is given.
• Figure 3, in numbers. The same configuration run twice: 26 pairs, median 0.53, from Sonnet 5 at 0.19 to GPT-6 Sol at 0.66. Two configurations from the same vendor: 298 pairs, median 0.39. Two configurations from different vendors: 342 pairs, median 0.34. Ringed: Fable 5.1 and GPT-6 Astra, both at high effort on the same brief, 0.48, with 57 defects shared, 41 found only by Fable and 20 only by Astra. The pairs come from 37 of the 38 runs; one read only one design.
• Figure 4, in numbers. Tier-A overlap medians: rerun 0.60 (26 pairs), same vendor 0.44 (298), different vendors 0.39 (342); among pairs whose tier-A lists are within five items of each other, same vendor 0.50 (118 pairs) and different vendors 0.50 (111). The left panel's per-run counts are in the table in A9.
• The verifier test. An earlier check had made same-vendor checking look useless: two of three Claude verifiers confirmed every item handed to them, 66 of 66 and 64 of 64. That is why the 134 flags of Figure 5 went to three verifiers.
• Figure 5, in numbers. Confirmed, refuted and cannot tell: Opus 5 verifier A 120, 12 and 2; Opus 5 verifier B, with a different brief, 110, 23 and 1; GPT-6 Astra on verifier A's brief 99, 17 and 18. Flags each pair disagreed on: A and B 20, A and Astra 25, B and Astra 34.
• Figure 7, in numbers. The list price of one review of all three designs, divided by the confirmed defects it raised; twenty-eight runs, the tick at each configuration's mean. GPT-6 Luna medium $0.001 (three runs), GPT-5.6 Luna medium $0.005 (three), Opus 5.5 low $0.03 (three), GPT-5.6 Terra medium $0.03 (three), GPT-6 Sol medium $0.03 (three), GPT-5.6 Sol medium $0.06 (three), GPT-6 Astra high $0.09 (two), Sonnet 5 high $0.10 (three), Opus 5 low $0.10 (one), Opus 5.5 xhigh $0.11 (three), Fable 5.1 high $0.23 (one). Nobody was billed these amounts. Grading adds $8 to $9 per run. The cheapest configurations also found the fewest, and effort labels are each vendor's own. The ten runs not priced are the four multi-agent team reviews, the two solo cells and four early single runs, for which no clean list price for one review of all three designs was worked out. The right panel's tier-A counts are in the table in A9.
• Cost scope. The $830 is the Claude side from 10 to 19 September at Claude Code's list-price meter; the reruns and the Opus 5 re-judge of 20 September are not in it, and the fifteen runs of 24 September came to about $240 at the meter, grading and auditing included. The GPT side was metered in Codex subscription points with no dollar total. Figure 7 prices each GPT run's tokens, from the Codex logs, at OpenAI list rates fetched 2026-09-19 and 2026-09-26, and each Claude run at the meter.
• The 287. The category coding ran on the pool before its last matching pass: 71 answer-key defects and 216 others, not the final 320.
• Blind coding. Two coders, Opus 5 and GPT-5.6 Sol, used the same codebook, the same blind presentation and the same order: exact agreement 85 percent, Cohen's kappa 0.83, with design against scoring the most common confusion. On the 194 defects beyond the key that both coders labelled alike, the cost gap is the same size, at a weaker family-wise p of 0.068 because the subset is smaller.
• Pins and planted defects. Every Codex run's model and effort were read back from its log. Where planted-defect results are recorded, in one audit pass and both re-judges, every planted defect was caught; as a verifier, GPT-6 Astra refuted one of six real planted defects and the first Opus 5 verifier none.
• Figure 2. The pipeline drawing was generated when the pool held 296 defects; it was cropped and its total relabelled by hand at 320.

Images: the hero and Figures 1 and 2 were drawn by GPT Image 2.5 Flare (gpt-image-2.5-flare) through the OpenAI API from Claude's prompts; the charts are hand-built SVG.