TL;DR
Luna had the higher observed planning mean and lower cost and latency than Sonnet 5 in the August 8, 2026 replay. Across 29 frozen conversations, scores were 0.731 versus 0.579, billing $0.0012 versus $0.1348 per decision, and median response times 8.6 versus 24.9 seconds. The evidence concerns this routing task; it supplies no pair-specific significance result for broader model superiority.
Planning models on 29 frozen conversations: published results
16-model campaign, 29 frozen conversations; 14 model rows shown in the public table. Published 2026-08-08. Each bar uses the original task scale. The original table retains all other dimensions and qualifications.
Show the table · 2 rows, 5 columns
Scroll sideways to compare all columns. Select a heading to sort.
| openai/gpt-5.6-luna | 0.731 | 0.068 | $0.0012 | 8.6s |
| anthropic/claude-sonnet-5 | 0.579 | 0.071 | $0.1348 | 24.9s |
No matching rows. Clear the filter to see all records.
16-model campaign, 29 frozen conversations; 14 model rows shown in the public table. Published 2026-08-08.
Which fits the tested job?
Luna had the higher observed planning mean and lower recorded bill and latency in this router replay. That makes it the practical candidate to try first for the tested task; there is no published pair-specific significance result proving broader superiority over Sonnet.
Planning models on 29 frozen conversations
The means were 0.731 for Luna and 0.579 for Sonnet, with standard errors 0.068 and 0.071. Historical cost was $0.0012 versus $0.1348 per decision, and p50 was 8.6s versus 24.9s. This comparison concerns routing the 29 frozen conversations, not coding-client features or every Claude workload.
Route a request and plan the work: follow the published task
A useful planner chooses the right kind of work before doing it. The complete customer conversations and system prompts remain private.
Published task summary
An incoming request and its existing conversation.
Choose whether to write a plan, call a catalog tool, ask for clarification or continue an existing agent.
Read what came back
A routing decision and plan. Every model receives the same saved conversation; the test compares that decision before fresh research begins.
Replay 29 frozen conversations across the model arms, then compare graded plan scores, billing and response time.
Results from tasks both models attempted
The tested model IDs are openai/gpt-5.6-luna and anthropic/claude-sonnet-5. Every result table below comes from a study in which both participated.
Compare the measured quality, historical billed cost and elapsed time within the same task and rubric.
Scores from different tasks are not combined into one winner score.
Planning models on 29 frozen conversations: read these measurements
Read the complete rubric, sample and billing notes
16-model campaign, 29 frozen conversations; 14 model rows shown in the public table
Route an incoming request: write a plan, call a catalog tool, ask for clarification or continue an existing agent.
Scoring
Top planning means were close at this sample size. The source's paired comparison with Opus was inconclusive.
Cost units
Cost is provider-billed dollars per decision. Prefix reuse can make it differ from list price multiplied by every input token.
Time units
p50 is median latency in seconds, not the duration of complete research.
Frozen conversation
A saved input kept identical across model arms so their decisions can be compared.
Standard error (SE)
An estimate of uncertainty around the sample mean. A small mean difference needs more evidence than the ranking alone.
No comments yet.