A September study of 21 model-harness pairs found Claude Code, Codex CLI and Pi finishing nearly the same tasks at very different cost, while October’s plan changes redrew which subscription can fund which tool. The contest between Claude Code, Codex, Pi and OpenCode is now about billing and trust more than benchmark rank.
Run the same model through different coding harnesses and its success rate barely moves, but the bill does. That is the central result of Arena’s HarnessTax study, published on 16 September 2026, which tested Claude Code, Codex CLI and Pi across 21 model-harness pairs. Claude Fable 5 solved 97.8% of SWE-bench Lite attempts in Claude Code and 96.7% in Pi, yet Claude Code averaged about $1.33 per task against Pi’s $0.67.
The study left out OpenCode, the fourth tool in this comparison, and it said nothing about who may pay for what. That second question moved faster than any benchmark. Since the start of October, Pi has shipped its 1.0 release, OpenCode has added a provider that runs Claude Code underneath it, Anthropic has renamed its two top plans, and OpenAI has cancelled a planned cut to its Pro allowance.
The thesis here is that Claude Code, Codex, Pi and OpenCode now compete on three things: what a task costs, which subscription can fund it, and how much of the harness the user controls. The evidence supports a narrower claim than “harnesses are equal”. On two open benchmarks the success gap was small and the cost gap was large, and subscription rules now decide who can exploit it.
Four tools, four routes to revenue
Claude Code is Anthropic’s agentic coding tool, and its documentation lists the terminal, VS Code, a desktop app, the web and JetBrains as surfaces. Most surfaces require a Claude subscription or an Anthropic Console account. Anthropic’s conditions for hosting it require the binary to stay unmodified, so the tool you run is the tool Anthropic published. That makes Claude Code the only product in this group where the harness, the model and the billing relationship belong to one company.
Anthropic’s weekly digest shows how fast that single-vendor stack moves. Claude Opus 5 became the default Opus model in Claude Code in the week of 20 July, with a 1M-token context window and a fast mode priced at $10 and $50 per million tokens. Claude Fable 5.1 followed in the week of 31 August, also with 1M tokens of context. The model and the harness ship together, which is the strongest argument for using the vendor’s own tool.
Codex sits in a similar position with a different license. OpenAI released the Codex CLI as open source on 16 April 2025, and the CLI carries an Apache 2.0 license while the desktop app is proprietary. Codex now runs through ChatGPT’s web app, the CLI, desktop apps for Windows and macOS and several IDE integrations, and OpenAI merged the Codex app into the ChatGPT desktop app on 9 July 2026. By March 2026 it had passed 2 million weekly active users. It is best understood as a feature of a ChatGPT plan rather than a product with its own price list.
OpenCode and Pi are the independents. OpenCode is MIT-licensed and built by Anomaly, which Y Combinator backs. Its CEO claimed about 13 million monthly users in a July interview, and the repository has passed 217,000 GitHub stars. Pi is MIT-licensed as well, is maintained by Earendil, and Earendil says hundreds of thousands of people use it every week.
Neither independent owns a frontier model, so both sell the freedom to switch. That freedom is the product, and it explains why their relationship with Anthropic and OpenAI is the most interesting part of the comparison. A shared habit eases the switch: Claude Code can read an AGENTS.md file in place of its own CLAUDE.md, and OpenCode generates AGENTS.md through its init command, so project instructions travel between the tools more easily than they did a year ago.
Pi 1.0 keeps four tools and reverses its MCP refusal
Pi is a minimal harness, and pi.dev still describes it largely in terms of what it left out. The core agent has four tools, which the HarnessTax authors list as read, write, edit and bash. Pi ships without sub-agents, plan mode, built-in to-dos, background bash or permission popups. Its answer to each gap is a file, a tmux session, a container or an extension. Pi asks users to build or install what other tools bake in.
Pi 1.0, released on 1 October 2026, bent that rule in places. Earendil added Codemode, which gives the model a JavaScript sandbox for composing tool calls, along with native MCP support, extension support for virtual models, deferred tool loading, cache warming for Anthropic models, mid-conversation system messages and a full-screen terminal interface by default. A post of 29 September explained the MCP reversal. Earendil now treats MCP as closer to OpenAPI with tool discovery, and it argues that the sandbox which suits MCP also suits Pi. In Pi, Codemode loads automatically when MCP is configured, and MCP tools are exposed to the sandbox in the way Codex does it.
The company released a second package the same day. Pi Durable is an experimental base for long-running agentic applications, and Earendil kept it separate to avoid growing Pi itself. The split shows the strategy: the coding agent stays small while a sibling package carries the larger ambitions.
Pi also works with subscriptions. It supports more than 15 providers through API keys or OAuth, according to pi.dev, and its sessions are stored as trees that can be branched and shared. The subscription question becomes central in the fifth section, because a harness that cannot use the plan you already pay for competes with one that can.
OpenCode 2 builds a server first and a terminal second
OpenCode’s distinctive choice is architectural. It runs as a local server, and the terminal interface, desktop app, IDE extension and SDK are clients of that server over HTTP. Three commands follow from the design: opencode serve runs it headless, opencode attach connects to a session on another machine, and opencode web adds a browser client. The server publishes an OpenAPI 3.1 specification that generates the official SDK, so a script can drive sessions through a supported API instead of faking keystrokes in a terminal.
The pace of change is the other defining trait. Version 2.2.0 on 27 October added experimental desktop plugins and removed the OpenCode GitHub Action. A day later version 2.2.1 added a /goal command that keeps a session working until a reviewer confirms the objective, opt-in scheduled prompts and image generation. Version 2.2.6 on 6 November added shared MCP connections and opencode run -w, which starts a scripted run in a new git worktree. The project also split its release lines. OpenCode 1 now stays on 1.x and the install script refuses to install 2.x, so moving to version 2 is a deliberate step.
Free software still has a price. OpenCode costs nothing under its MIT license, but Analytics Vidhya’s September guide lists the routes to a model: your own API keys, the pay-as-you-go Zen gateway, the Go subscription for open-weight models, a Copilot or ChatGPT login, or local models through Ollama. The same guide warns that the permission system is a workflow safeguard rather than a security sandbox, and that small local models make more invalid tool calls and miss more links between files.
The combination of a server, a documented API and a plugin system gives OpenCode the shape of a platform. That is a different bet from Pi’s, which keeps the agent small and lets extensions grow around it, and from Claude Code’s, which grows the product itself across terminal, IDE, desktop and web.
The 21-pair study puts a price on the harness
The HarnessTax authors fixed what most comparisons leave loose. They ran the same 30 randomly sampled tasks from SWE-bench Lite and Terminal-Bench 2.0, repeated each task three times, used each harness’s high-effort setting, capped every attempt at 100 agent turns and priced all tokens from one direct-API price list dated 1 September 2026. Seven models and three harnesses produced 21 pairs.
Success barely moved. The average effect of the harness stayed within 2 percentage points on SWE-bench Lite and within about 5 on Terminal-Bench. Cost did move. Across shared models, Claude Code cost about 2.0 times as much as Pi and 1.6 times as much as Codex on SWE-bench Lite, and about 1.5 times as much as Pi on Terminal-Bench 2.0. For Claude Fable 5, Pi and Claude Code averaged 15.4 and 15.3 turns per attempt, so the gap sits in spending per turn rather than in turn count.
The authors point to the first model call. Claude Code’s mean initial context across all seven models was more than 10 times Pi’s, because of longer instructions and larger tool schemas. They caution that caching, generated tokens and later calls also drive spending, so the context gap explains part of the bill and not all of it.
A second finding cuts against provider loyalty. Across the six Anthropic and OpenAI models and both benchmarks, an alternative harness produced the highest observed success rate in nine of twelve comparisons. Sonnet 4.6 solved 68.9% of SWE-bench Lite attempts in Codex against 66.7% in Claude Code at similar cost. The authors read this as evidence that a model’s capabilities carry over to other harnesses, which weakens the case for treating the vendor’s own tool as the default.
Two pairs from the study
| Model and benchmark | Harness | Success rate | Average cost |
|---|---|---|---|
| Claude Fable 5, SWE-bench Lite | Claude Code | 97.8% | $1.33 |
| Claude Fable 5, SWE-bench Lite | Pi | 96.7% | $0.67 |
| GPT-5.6 Sol, Terminal-Bench 2.0 | Pi | 83.3% | $0.42 |
| GPT-5.6 Sol, Terminal-Bench 2.0 | Codex CLI | 78.9% | $0.76 |
The rows come from the HarnessTax text and show the pattern in two places: near-equal success at roughly double the cost for Claude Code, and a higher success rate at about half the cost for Pi against Codex. The two benchmarks use different tasks, so compare rows only within each model.
Subscriptions split Anthropic from OpenAI
Anthropic’s position is written down. Claude Code’s legal page says OAuth sign-in is for people who bought a Claude plan and is designed for ordinary use of Claude Code and other native Anthropic applications. Developers building products, including on the Agent SDK, should use API keys. The page says Anthropic does not permit third-party developers to offer Claude.ai login or to route requests through Free, Pro or Max credentials on behalf of users, and it reserves the right to enforce that without notice.
The same page leaves one door open. It says nothing prevents an end user from signing in to the unmodified Claude Code binary with their own subscription, including where a platform hosts that binary. OpenCode’s 30 October Claude Agent provider walks through that door. According to CellCog’s analysis of the provider spec, OpenCode runs the user’s installed claude program through Anthropic’s Agent SDK, swaps in its own tools and never reads or replays the saved login. CellCog also notes that Anthropic’s SDK documentation still bars claude.ai login in third-party products unless approved, and that it found no public Anthropic statement about the provider. Whether Anthropic tolerates it is unresolved.
Anthropic’s help center adds a second constraint. Its note of 7 October says that when you sign in with a Claude plan, the Agent SDK and claude -p still draw from subscription limits, and that Max, Ultra and Team plans now include monthly API credits usable with a Console API key. An earlier plan to move SDK usage onto a separate credit was paused on 15 June.
OpenAI took the opposite posture in public. Simon Willison reported a 30 March post from OpenAI’s Romain Huet saying people should be able to use Codex and their ChatGPT subscription in the app, the terminal, JetBrains, Xcode, OpenCode and Pi. OpenAI’s Codex for Open Source page, which offers maintainers six months of ChatGPT Pro with Codex, tells developers to code in the tools they prefer and names OpenCode, Cline, pi and OpenClaw alongside Codex.
Plan allowances changed twice in the last days of October
Anthropic’s change arrived on 30 October. Max 5x became Max and Max 20x became Ultra, with prices unchanged at $100 and $200 a month. Verified Ultra accounts lost the limit that resets every five hours, while weekly limits stayed. The Next Web reports that Anthropic’s pricing page now lists Ultra at 10 times Pro usage per week, where the old name implied 20 times, and that Anthropic’s announcement did not mention the difference. Both plans include Fable, Anthropic’s top model, for up to half of the weekly limit, and carry monthly API credits of $100 and $200.
OpenAI went through a reversal in the same days. A September 29 email had announced a reduction in the Work and Codex allowance on Pro 200. OpenAI’s help page now says that reduction is cancelled, according to Mixed. Existing holders of the higher Pro 200 allowance keep it, holders of the lower allowance had it doubled to match, and Pro 100 and Pro 500 allowances doubled as well. The same report notes that OpenAI publishes no simple count of the tasks an allowance buys, so doubled usage does not promise doubled finished work, and that faster modes can drain the allowance sooner.
Read together, these changes show that flat-rate plans are the real product and that allowances are tuned in public, sometimes within days. That is an editorial inference, but it follows from the pattern. A developer who selects a harness for its subscription compatibility is buying exposure to those adjustments. A bring-your-own-key setup in Pi or OpenCode avoids that exposure but pays API rates, and this is where the HarnessTax cost gap turns into money: the harness that sends fewer tokens per task stretches the same budget further.
Anthropic’s earlier enforcement shows how quickly the ground can shift. The Next Web recalls that in April Anthropic blocked OpenClaw and other third-party agent tools from using Claude subscriptions. Six months later, OpenCode’s Claude Agent provider shows how third-party tools adapt: they move closer to the official binary instead of imitating it.
The study has limits and the incidents are real
The HarnessTax authors name their own limits. Both benchmarks are open, so models may have met the tasks in training, and results may differ on other workloads. Thirty tasks per benchmark is a small sample, OpenCode was absent, and Pi ran with two added packages for subscription keys and turn control. Claude Code’s extra weight may also buy things the benchmarks do not score. Its documentation advertises hooks, skills, parallel background agents and cloud scheduled tasks, and auto mode became the default permission mode for new Pro, Max and Team sessions on 14 August. The study measures cost and success on short tasks, so it cannot say which tool wins a three-day refactor.
Open tooling carries its own risks. OpenCode disclosed on 17 October that a breach of its hosted Console had affected 1.4% of Console users and may have exposed provider keys, with co-founder Dax Raad stressing that the incident involved the Console and not OpenCode itself. Pillar Security separately reported a critical remote code execution flaw in OpenCode versions 1.0.137 to 1.18.32 on Windows and WSL. Anomaly patched it in version 1.18.33 on 28 September without a CVE or advisory, after saying malicious MCP servers were outside its threat model. Pillar argued that scanners would not flag affected installations, and an attacker-controlled MCP server was enough to run code during an OAuth handshake.
Codex is not exempt. Researchers found a vulnerability in March 2026, since patched, in which malicious GitHub branch names could inject commands during task setup and expose GitHub tokens. Pi answers the permission question differently: its site says to run in a container or build your own confirmation flow instead of relying on popups. Every harness in this comparison executes model-chosen commands on a developer’s machine. The differences lie in defaults and disclosure, not in whether the exposure exists, and a team adopting any of them needs a sandbox policy of its own.
Choosing by subscription, governance and your own task set
A developer who already pays for ChatGPT has the clearest path. Codex CLI is the first-party route, and OpenAI has publicly welcomed Pi and OpenCode on the same subscription, so trying either costs a login. The study suggests running Pi on the same model before assuming Codex’s own harness is the cheapest option.
A Claude-first developer should treat Claude Code as the supported way to use a Claude subscription and should treat OpenCode’s Claude Agent provider as unapproved until Anthropic says otherwise. Anyone who wants Claude inside OpenCode, Pi or another harness has a clean alternative in API keys. Monthly API credits of $100 on Max and $200 on Ultra now offset part of that cost, but the credits apply to Console API-key usage and do not change the rules for subscription sign-in.
A team lead choosing for several engineers faces a different question: who answers when a key leaks or a harness ships a fix without an advisory. Claude Code and Codex come with a vendor to contract with. OpenCode and Pi are MIT-licensed and move faster, and OpenCode’s silent patch shows what that speed can cost in disclosure. Governance, not benchmark score, should decide the team’s shortlist.
A cost-sensitive builder should measure instead of trusting any ranking, including this article’s. The study offers a method as well as a result: fix the model, take a few dozen of your own tasks, run each through every candidate harness three times and compare cost beside success. Evidence that one tool costs twice as much for the same result is cheap to gather and specific to your codebase.
Convergence in features, divergence in permission
The evidence supports a split verdict. On cost, minimal harnesses hold an advantage that benchmarks can measure, and the large vendors have begun borrowing from them and from each other. Pi added MCP, OpenCode added /goal, and Claude Code had already added its own /goal command in the week of 11 May, which lets Claude keep working across turns until a completion condition is met. Features move in every direction, so feature lists will keep converging.
What will not converge is permission. Who may bring which subscription to which harness is a policy choice, not an engineering one, and policy is set by the two companies that sell the models. OpenAI has chosen openness for now, and that openness runs through open-source programs and through the Codex backend that third-party tools call. Anthropic has chosen a line that its own documentation draws narrowly, and OpenCode has found a path along that line which Anthropic has not publicly endorsed.
Three developments would change this judgment. A public Anthropic statement approving or blocking OpenCode’s Claude Agent provider would settle the largest open question. A benchmark that includes OpenCode and long, multi-session work could show that Claude Code’s heavier context buys success the short tasks cannot see. A tightening of OpenAI’s terms for third-party harnesses would erase the main reason to reach Codex’s backend through Pi or OpenCode. Until one of those happens, the sensible default is the cheapest harness your subscription allows, verified on your own tasks.
Questions readers ask about the four coding agents
In the HarnessTax study it was. Across shared models, Claude Code cost about 2.0 times as much as Pi on SWE-bench Lite and about 1.5 times as much on Terminal-Bench 2.0, while average success differed by only a few points. The study used 30 tasks per benchmark and two open benchmarks, so test your own tasks before acting on it.
OpenCode 2.2.3 added a Claude Agent provider that runs your installed Claude Code and accepts a subscription, an API key or a cloud account. Anthropic’s documentation bars third-party developers from offering claude.ai login unless approved, and CellCog found no public Anthropic statement about the provider, so the position is unresolved. An API key is the route Anthropic’s documentation names for developers.
OpenAI’s Romain Huet said in March that people should be able to use Codex and their ChatGPT subscription in tools including OpenCode and Pi, and OpenCode’s changelog lists a Sign in with ChatGPT option. Check current terms before relying on it, because plan allowances changed in October.
The Codex CLI is open source under the Apache 2.0 license. The desktop app is proprietary, according to Wikipedia’s summary of the project.
Yes, since Pi 1.0 on 1 October 2026. Earlier versions declined MCP, and MCP is now built in through Codemode, which loads automatically when MCP is configured. MCP extensions still work as well.
OpenCode said the breach hit its hosted Console and affected 1.4% of Console users who had external providers linked. Co-founder Dax Raad said it involved the Console, not OpenCode itself. Anyone who stored provider keys in a Console workspace should revoke and replace them.
The software is MIT-licensed and free. Model access is not: you pay through your own API keys, the Zen gateway, the Go subscription, an existing Copilot or ChatGPT login, or the hardware cost of running local models through Ollama.
Pi Durable is an experimental MIT-licensed package released with Pi 1.0 as a base for long-running agentic applications. Earendil kept it separate from Pi itself so the coding agent stays small.
Not in the end. OpenAI announced a reduction on 29 September, then cancelled it, according to Mixed. Existing holders of the higher allowance keep it, and holders of the lower one had it doubled to match.
No single tool wins in the available evidence. The HarnessTax study found similar success rates across Claude Code, Codex CLI and Pi on its tasks, with an alternative harness posting the top success rate in nine of twelve comparisons, and it did not test OpenCode.
Author:
Jan Bielik
CEO & Founder of Webiano Digital & Marketing Agency
This article is an original analysis supported by the sources cited below
HarnessTax: How Much Does the Harness Matter for Coding Agents?
Arena’s 16 September 2026 study of 21 model-harness pairs across Claude Code, Codex CLI and Pi, source of every cost, success and context figure in this article, including its stated limits.
Pi 1.0
Earendil’s 1 October 2026 announcement, which supplied the Pi 1.0 feature list, the weekly-user claim, the MIT license and the introduction of Pi Durable.
“You Said No MCP!”
Earendil’s 29 September 2026 explanation of why Pi added MCP, how Codemode works and why MCP now loads through a JavaScript sandbox.
Pi Coding Agent
The official Pi site, used for the list of features Pi leaves out, its provider count, session trees and its container-first stance on permissions.
OpenCode changelog
OpenCode’s own release notes for versions 1.19 to 2.2.6, used for the 2.x timeline, the Claude Agent provider, the removed GitHub Action and the Sign in with ChatGPT entry.
OpenCode Explained: The Open-Source AI Coding Agent
Analytics Vidhya’s September 2026 guide, used for OpenCode’s client-server architecture, its routes to a model and the caveats on permissions and local models.
OpenCode Breach Could Affect OpenAI, Anthropic and Google
Infosecurity Magazine’s 19 October 2026 report on the Console breach, including the 1.4% figure, the user and star counts and the Console-versus-OpenCode distinction.
Researchers Warn OpenCode Quietly Patched Critical RCE Flaw
Infosecurity Magazine’s 20 October 2026 report on Pillar Security’s remote code execution finding, affected versions, patch date and the absence of a CVE.
Claude Code legal and compliance
Anthropic’s own rules on OAuth, API keys, third-party login and hosting the unmodified Claude Code binary, the basis for the subscription analysis.
Use the Claude Agent SDK with your Claude plan
Anthropic’s help article, with the 15 June pause and the 7 October update on subscription limits and monthly API credits.
Claude Code overview
Anthropic’s documentation of Claude Code’s surfaces, subscription requirements, third-party provider support, AGENTS.md compatibility and automation features.
What’s new in Claude Code
The Chinese-language edition of Anthropic’s weekly digest, used for the May introduction of /goal and the 14 August default for auto mode.
OpenCode Brings Back Claude Subscriptions via the Agent SDK
CellCog’s 30 October 2026 analysis of OpenCode’s Claude Agent provider, its design and the unresolved question of Anthropic approval; a vendor blog treated as secondary.
Anthropic renames Claude Max plans and drops Ultra’s five-hour limit
The Next Web’s 1 November 2026 report on the Max and Ultra renaming, the plan prices, the weekly limits, API credits and the April blocking of third-party agent tools.
OpenAI reverses its ChatGPT Pro 200 usage cut
Mixed’s 1 November 2026 report on OpenAI cancelling the planned Pro 200 reduction and doubling Work and Codex allowances on some tiers.
A pelican for GPT-5.5 via the semi-official Codex backdoor API
Simon Willison’s 23 April 2026 post, which records Romain Huet’s 30 March statement that ChatGPT subscriptions should work in OpenCode, Pi and other tools.
Codex for Open Source
OpenAI’s program page, used for its offer of ChatGPT Pro with Codex to maintainers and its naming of OpenCode, Cline, pi and OpenClaw as welcome tools.
OpenAI Codex (AI agent)
Wikipedia’s overview of Codex, used for the April 2025 CLI launch, the licensing split, the July 2026 app merger, user growth and the March 2026 branch-name vulnerability.
| Citing this article? Brief excerpts are welcome. Please credit Webiano.digital, name the author where stated, and include a link to https://webiano.digital and to this original article. Full or substantial republication requires prior written permission. Read our Copyright and Content Use Policy. |
This article was prepared with the assistance of artificial intelligence tools. The content underwent expert human review, and Webiano Digital & Marketing Agency assumes editorial responsibility for its final version and publication.
